Repository navigation
S3 Files Uploaded by Parse Server Inaccessible to api.parse.com clients #1582
Description
We migrated our database before the deadline, it is working fine,
We are hosting now ParseServer on our server and want to use it, we will release a new version of our iOS App that points to our ParseServer.
People using the new version of the App won't see any problem, but the user who still using the older version, won't be able to see the new data stored on our own S3.
Are you going to release a tool that can solve this problem ? something like configuration in the dashboard where we can configure a Storage Adapter?
Files Migration is in itself no problem, we can go through all files and copy them to our own S3, it is all about users who use our old version can see the objects uploaded to the new S3.
How fast do you think we are going to see this problem solved?
Activity
flovilmart commentedon Apr 21, 2016
I'm not sure (and I may be wrong) about that problem. Your newly stored files should be publicly accessible and the URL's for the new files on your S3 will be stored in your DB alongside the old ones. So given that, as long as your files are properly uploaded to S3, an 'old' client would just get the new URL, pointing to your S3. Is it an issue you're seeing now?
benishak commentedon Apr 21, 2016
The current situation now, the users who are using the app that point to parse.com are not able to see files stored in our DB or our S3 because the look like this
http://files.parsetfss.com/$filekey/$filenameand not like this
https://api.parse.com/files/$appid/$filename // if file is stored in the DBor am I wrong?
flovilmart commentedon Apr 21, 2016
that should not matter as the File objects store the full URL. Are your newly stored files accessible via your web browser?
benishak commentedon Apr 21, 2016
no because it forward me to http://files.parsetfss.com
flovilmart commentedon Apr 21, 2016
for the new files? How did you configure the S3 Adapter?
benishak commentedon Apr 21, 2016
The old App uses : api.parse.com
The new App uses: api.ourparseServer.com
People using the "new App" can see files hosted on our S3 because we configured our S3Adapter and also the files hosted on Parse S3 ('ttfs')
But lets assume not all users upgrade the App so some users who use OLD APP, which use
api.parse.comare not able to see or access object upload to our OUR S3hramos commentedon Apr 21, 2016
This is a known issue. api.parse.com has no knowledge of your specific Parse Server configuration, it only knows about files.parsetfss.com and files.parse.com. If Parse Server stores only the filename, api.parse.com will assume it is being hosted on files.parse.com and it will return a files.parse.com URL, which expectedly will fail to load the resource.
flovilmart commentedon Apr 21, 2016
Don't we store the full URL if not, that's a big deal
hramos commentedon Apr 21, 2016
My understanding is that only a filename is stored in the database, and it is up to the file adapter to look up the static asset or construct the direct URL when direct access is enabled.
See https://github.com/parse-server-modules/parse-server-s3-adapter/blob/master/index.js#L144 as reference.
benishak commentedon Apr 21, 2016
Ok is Parse.com working on any solution or a tool on the dashboard that allow us to configure our own S3 or any kind of solutions? If yes how long will it take to see something?
[-]S3 Migration[/-][+]S3 Files Uploaded by Parse Server Inaccessible to api.parse.com clients[/+]hramos commentedon Apr 21, 2016
We're heads down on database migration tasks at the moment. Improving the file migration story is something we have on our sights as we enter Phase 2 on April 28. No promises but this seems something that would be useful to look into.
See #8 and #1521 for related issues.
flovilmart commentedon Apr 22, 2016
That Would be fairly Easy to make sure we store the full URL's. I believe the controller asks the adapter for the custom location, so it should in that case be stored as a full URL. I'll have a look.
kvreem commentedon May 17, 2016
@benishak any luck?
kvreem commentedon May 18, 2016
When using the default file grid adapter or amazon s3 older versions of the app cannot see the newly uploaded files. Users on new versions can see both. Did anyone figure a workaround or managed to get it working on grid adapter? Was surprised when the default adapter wasnt working either.
6 remaining items
kvreem commentedon Jun 1, 2016
@kkgelu is it possible to just add a prefix automatically so that old clients can see the new files?
kkgelu commentedon Jun 1, 2016
@kkhattab the clients, old or new, just go and fetch a file a from its url. The url, as I understood is constructed by the server (I have not looked into the implementation of ParseFile), it's the old server/api.parse.com couldn't understand the new files (the mongodb only keeps the names) and give its clients the wrong urls. (I'll make a concrete example next month when we get back to this issue again)
JeremyPlease commentedon Jul 12, 2016
I've run into this issue while migrating a relatively large app.
The issue is that Parse Server only stores the filename and not the full url.
@flovilmart, @hramos If I submitted a PR to update Parse Server (and/or file adapters) to store the full urls, could api.parse.com be updated to handle it accordingly (ie. files without full url get
urlset to files.parse.com and files with full urls use those instead)?flovilmart commentedon Jul 12, 2016
I believe api.parse.com won't be updated now as it's winding down. We should aim for a solution that would work correctly, not expecting the original api would change to accommodate the update.
JeremyPlease commentedon Jul 12, 2016
@flovilmart @hramos
If api.parse.com won't be updated then there is no possible way (that I see) for it to return the proper url for files uploaded to Parse Server.
The only possible solution I see is to write a file adapter to proxy file upload requests to api.parse.com/1/files. That way all files are stored in Parse's bucket and url will be set properly from both hosted and open source Parse.
Any other ideas or solutions you can think of?
hramos commentedon Jul 12, 2016
I can't comment on whether
api.parse.comwill be updated to handle files uploaded by clients using Parse Server at this time. What I can say is that this issue only affects old clients that useapi.parse.com, therefore the issue can be mitigated by updating the app to the newer version that uses Parse Server. We're almost 50% of the way through the sunset period, so I'd encourage everyone to focus on getting that updated version of the app out ASAP.This issue seems to be a duplicate of #8. Can we close this issue and continue the thread there?
JeremyPlease commentedon Jul 12, 2016
@hramos I definitely agree with getting updated versions of apps out ASAP. I'm just trying to mitigate the issue of users on old app versions (before they upgrade) unable to access files uploaded through Parse Server.
If there's no recommended solution for this, then I can accept it as a "migration side effect." We should document it somewhere with something like the following:
After we document this, we can close this issue 😃
hramos commentedon Jul 12, 2016
@JeremyPlease it is mentioned under Files in the Migration Guide:
JeremyPlease commentedon Jul 12, 2016
Thanks @hramos. Sorry I missed that in the migration guide!
I think it's fair to close this issue now and continue discussion of migrating legacy files on #8.
acinader commentedon Jul 19, 2016
@hramos should we expect that there will be no solution to support parse.com and parse-server file access simultaneously?
At the risk of being obvious ;) it would be super nice not to break parse.com immediately when pushing the parse-server client.
Since addressing this issue would require a change to parse.com, I'm asking here so we can plan accordingly. We'll have to deal either way, but would be interested to know if a decision has been made already?
The "as of this writing" is giving me hope, so better to dash those hopes now if necessary (in which case the readme should probably be updated too?)
hramos commentedon Jul 19, 2016
@acinader I can confirm that there are currently no plans to update the legacy parse.com service.
sud80 commentedon Aug 3, 2016
@hramos Then essentially are we saying that, the migration guide that has been provided where we can support running older clients till end of January without breaking functionalities is gone for a toss. As you can see here in step 12 of migration guide https://www.parse.com/migration, the new clients and old clients work together for a period of time.
I don't understand why facebook is taking all its customers for granted. This is really sad state of affairs.
hramos commentedon Aug 3, 2016
@sud80 the process of migrating an app is non-trivial and there will be some bumps in the road for some apps. The earlier you release an updated app that uses something other than api.parse.com, the earlier your user base will be able to transition and get past these bumps.
sud80 commentedon Aug 3, 2016
@hramos I do understand it is non-trivial but the point is to see if there is a way to avoid customer impact. Even however early I move, the way it is currently, the existing users with old app is going to get affected. But yeah we are left with no choice.