How to make a video streaming site
Thursday, July 3rd, 2008
The video streaming application setup based on my assumptions to explain what a developer needs to consider from the technical/scalability point of view to create video streaming social media website such as YouTube, Break.com or Google video.
Video precessing
Conversion of various uploaded video file types to the FLV (Flash video streaming format)
Mature video conversion classes are available in public license for Python programming language. Pyhton or C will make the processing more real-time although a processing queue is very likely to happen. Resulting FLVs are stored on a permanent storage device.
Video precessing server certainly needs a massive processing power.
Application server
Django/RoR or other web frameworks will be useful at this point. I would prefer to use Django framework. There is a built in object caching and support for memcached. Heavy load is to be expected as a nature of the video streaming sites and caching at various levels of the process is very handy.
This needs to be engineered to expect load on the certain videos, while other videos will have hardly any views. Caching will make sure that the pages for videos under heavy load are served instantly, freeing up the resources on the machine.
Database server
Running MySQL or PosgreSQL instance serving user posts and and comments controllers’ textual data requested from the website processing server.
Static content server
Serving thumbnails, cached pages, streaming video files
Nature of video files is that they are big, therefore this machine(s) will need a some disk space, separate from the three machines above. Nginx is a good hint for serving video thumbnails. This server will also serve the JavaScript, CSS files, Flash action scripts + design related imagery. This should also account for the scheduled backups to a tape storage device as the project grows (not necessarily at the start).
See Nginx proxy to the application server for a good hint on serving static content.
Database server, website precessing & video precessing and static content parts will need to be loosely coupled allowing the developer to separate them and replicate setups on more physical machines as the site grows.
So what do you think?
Let's do business: 00447 857 309 211 / vizualbod@vizualbod.com