Repository navigation
S3 File adapter - accessing previous, non-s3 files #6273
Description
We have a mobile app that runs on Parse Server 2.8.4, hosted at DigitalOcean. We were using the default file adapter (GridStoreAdapter) and now we want to use Spaces by DigitalOcean. This is our updated index file:
var express = require('express');
var ParseServer = require('parse-server').ParseServer;
var ParseDashboard = require('parse-dashboard');
var path = require('path');
var S3Adapter = require('parse-server').S3Adapter;
var AWS = require("aws-sdk");
var databaseUri = process.env.DATABASE_URI || process.env.MONGODB_URI;
//Set Digital Ocean Spaces EndPoint
const spacesEndpoint = new AWS.Endpoint('fra1.digitaloceanspaces.com');
//Define S3 options
var s3Options = {
bucket: 'bucketName',
baseUrl: 'https://domain.com',
region: 'fra1',
directAccess: true,
globalCacheControl: "public, max-age=31536000",
s3overrides: {
accessKeyId: 'accessKey',
secretAccessKey: ' secretAccessKey',
endpoint: spacesEndpoint
}
};
var s3Adapter = new S3Adapter(s3Options);
var api = new ParseServer({
databaseURI: databaseUri,
appId: process.env.APP_ID,
allowClientClassCreation: false,
masterKey: process.env.MASTER_KEY,
fileKey: 'fileKey',
serverURL: process.env.SERVER_URL,
publicServerURL: 'http://ip/parse',
filesAdapter: s3Adapter
});
Everything works great! The files are stored and retrieved properly, except of the old ones! Every image that was stored with the GridStoreAdapter cannot be fetched anymore! Is there anything that can be done in order to be able to view both old and new files?
Activity
[-]S2 File adapter - accessing previous, non-s3 files[/-][+]S3 File adapter - accessing previous, non-s3 files[/+]dplewis commentedon Dec 11, 2019
A PR was submitted just for this purpose.
parse-community/parse-server-s3-adapter#79
I'll do a release right now so that you use it without having to update your server.
You could use parse-files-utils to transfer the files.
itssotsot commentedon Dec 11, 2019
Thank you very much @dplewis ! So what should I do now? I'm kinda lost!
dplewis commentedon Dec 11, 2019
Wait for the 1.4.0 release of the adapter.
Remove
var S3Adapter = require('parse-server').S3Adapter;Add
var S3Adapter = require("@parse/s3-files-adapter");https://github.com/parse-community/parse-server-s3-adapter#usage-with-digital-ocean-spaces
itssotsot commentedon Dec 11, 2019
And after that, it will know where each file is? MongoDB or DO Space?
dplewis commentedon Dec 11, 2019
Parse Server doesn't currently allow multiple file adapters. Therefore, I recommended the migration tool.
itssotsot commentedon Dec 11, 2019
So wait, I'm confused! Do I have to transfer all the files from Mongo to the DO Space? Wouldn't that PR you mentioned take care of it? (sorry for bothering you, but I really have no idea of how all these work)
I mean wouldn't it separate the MongoDB files and the DO files?
dplewis commentedon Dec 11, 2019
A fileAdapter is used to access the files. Since you changed GridStoreAdapter, you can't use it to fetch files anymore from Mongo.
Yes they will be separated. You could use a workaround to access the mongo files.
itssotsot commentedon Dec 11, 2019
And that would be the PR you're going to release?
dplewis commentedon Dec 11, 2019
@Samigos Sorry I reread your post and misunderstood. The PR #6274 was to support s3 overrides and endpoints but looks like you have that working.
You have to migrate your files if you want to view old and new files.
itssotsot commentedon Dec 11, 2019
Ok! Thank you very much!
itssotsot commentedon Dec 12, 2019
Hey @dplewis once more 😄I'm sorry for bothering you again, but I'm desperate! I'm stuck in the migration process and there's really not much I could find!
The repo you sent me above, parse-files-utils can it be used to transfer my files to an S3 bucket? If so, how exactly? I started following the guide, but as soon as I hit
npm start config.jsit failed with the error logs below:Any ideas?
dplewis commentedon Dec 12, 2019
That is an old repo. I’d recommend forking it and make changes you need and run it.
The core logic is there on how to get the all files from the Schema and transfer them.