View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0000159||Composr||core||public||2010-04-10 18:57||2016-06-26 22:50|
|Reporter||Chris Graham||Assigned To|
|Fixed in Version|
|Summary||0000159: Google AppEngine support|
|Description||(See notes below, original description no longer valid)|
|Time estimation (hours)||70|
AppEngine now supports native PHP. There are some restrictions/quirks, however. You cannot write to the filesystem (well you can, but it must be partitioned - and you cannot write PHP code), and it's not a normal webserver - so custom request handling is required. It would be good to implement support for memcached and task queues.
||Lots of limits still: https://developers.google.com/appengine/docs/php/runtime#Function-Support|
I'd love to implement this, but there are very serious practical hurdles right now. It's not really just a matter of designing Composr around the appengine limits, it's also a matter of designing a website with appengine in mind and having a developer workflow to maintain that website.
Major issues include:
- you cannot dynamically install new PHP files, so to add/remove addons or perform upgrades would be problematic (WORKAROUND: Produce an alternative way of specifying addons, such that files may remain when an addon is not installed)
- Uncertainties about cost. The appengine prices went up a while back, and it may well be a lot cheaper just to buy dedicated servers. (Then again, you can get free hosting on appengine for low-load sites, which is great)
- No custom PHP setup, e.g. your own choice of extensions, or additional backend services such as ffmpeg
- No server command line, you have to go through web interfaces to access your files/data
Lesser issues we would need to workaround:
- 32MB upload limit (stops videos, large downloads). You can workaround with the 'blob store', but it requires a custom uploader implementation
- A new installer would be needed (probably a local tool that integrates with the push system)
- A new upgrader would be needed (ditto, with some kind of 'system profile' generator to drive it)
- Refactoring a lot to use tasks
- no long running pages, without rewriting them to use the task system
- no large downloads, so things like CSV download are questionable; we'd need to generate in the background then email a link
- 1000 file in a directory limit, meaning we'd need to change how templates work
- Separate writable files from PHP code files
- Replacing .htaccess with an app.yaml file
- Implement memcache as a persistent cache backend (as APC etc caching not support)
- Must manually enable: phpversion, parse_str
- Must workaround missing: disk_free_space
- The fake filesystem does not actually properly support directories (although you can put stuff in directories, they are implicit within file paths used, not explicitly existent)
- Must workaround missing: escapeshellarg, escapeshellcmd, exec, shell_exec
- Must workaround missing: fsockopen
- Must workaround missing: tmpfile, tempnam
- No mbstring (i.e. botched unicode support)
- Use of Cloud SQL means higher cost than datastore, without getting the performance of non-SQL's eventual consistency; yet using datastore would mean a significant effort (see the original description of this task)
It hurts that it is so developer-centric, because I think there's a great deal of synergy between App Engine and Composr. I love the approach. The 'Major issues' though just stop it being useful to the typical Composr user, and the lack of support for certain key things mean that larger projects need many decisions made around appengine's requirements.
Philosophically, this is why I love AppEngine...
The constraints it places actually help to guarantee the scalability it is built for. It forces you to do things the right way, then you reap the benefits. It's also pretty simple, you don't need to manage servers or configure infrastructure anymore, so long as you make sure you put your design decisions in line with the philosophy.
Composr has always been engineered to embrace these kinds of sensible rules, and adding more is fine if we can derive benefit from it. We can do it in this project because it is centrally designed, as opposed to other community-developed CMSs, where it is hard to enforce new standards because too many people have to be involved to make it work.
This is now implemented, but unfortunately cannot be tested due to a critical bug in GAE (a segment violation memory error in the production environment). Google are looking into it, and we're stuck until they finish. I'll therefore have to put it on freeze. It has led to some general improvements in Composr, so was not such a waste, but it's a bit frustrating.
The remaining todo list is...
Test live generally
Test live performance (any latency issues with storage remaining?)
Test automated admin login