Yesterday I watched in Comcast, the movie “The Pursuit of Happyness“. Will Smith has superbly portrayed the character of a struggling salesman who takes custody of his son as he’s poised to begin a life-changing professional endeavour. The story is inspired by a true story of an investment tycoon (of the same name portrayed) Chris Gardener. An emotional story told very well, the boy (done by Will Smith’s real son Jaden Smith) has acted great especially in the scenes – where they are moving from place to place after a long day when they don’t have a place to sleep.
In this PDC2008 talk, Chuck Lenzmeier – the Architect in Azure team explains how the Virtual Images of OS works in the Azure cloud data centers. His Bio in PDC2008 says that Chuck has been with Microsoft since 1989, and worked on all versions of Windows from NT 3.1 to Vista. The below video really helps to understand how the Virtual Images are being managed to achieve Windows Azure manage while retaining absolute compatibility with existing OS, Software and Applications.
His co-speaker Frederick Smith from Microsoft explained the other aspects of Windows Azure.
Windows Azure provisions and monitors hardware elements (Compute nodes, L2 Switches, LBs, Routers), hardware lifecycle management (burn in tests, diagnostics and repair, failed hardware are replaced) and capacity planning
Azure Fabric is highly available: Network has redundancy built in, services deployed across fault domains, load balancers route traffic to active nodes only, Fabric Controller state check-pointed periodically, FC state is stored on multiple replicas
PDC 2008 CTP release of Azure has Automated Service deployment, Three Service templates, change number of instances, simple service upgrade/degrade, Automated service failure discovery and recovery, external VIP Address/DNS name per service, Service network isolation enforcement & automated hardware management
For 2009 release will have ability to model more complex application, richer life-cycle management & richer network management
I was in the talk by Andrew Fitzhugh from HP’s Magcloud.com. The site is HP’s attempt to do on demand publishing of magazines and allow anyone to publish, sell and distribute magazines. An interesting statistics that was shown that in USA about 3.6Billion magazines were delivered to US News stands last year and out of which 2.3Billion was never read.
He talked about how Magcloud moved their front-end systems and portions of storage to Windows Azure.
Magcloud.com on-premises architecture
Magcloud.com utilizing Windows Azure
Seeing the title I had good expectations, but the session turned out to be disappointing. The speakers completed the talk in 30 minutes and didn’t have anything more to talk or go into architecture or code details. The project seemed to me to a half-hearted attempt to test drive Windows Azure and was a simple project to showcase anything interesting.
Today was the keynote by Rick Rashid, Sr. VP from Microsoft Research. Notes on the session:
Why is fundamental research important to a company like Microsoft or country like USA. It is to survive tough times, about agility through your earlier investments and having smart people
Talked about Terminator, liveness property. Basically if you create a lock in say “C” will the code release it and so on
Talked about Dryad – Dryad is an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs, without knowing anything about concurrent programming
The study on computer “programs” help us to understand more on human “cells”, similarly study on “cells” is helping us to understand “programs”
Microsoft has been working with Washington university for Collaborative technologies – ConferenceXP
Microsoft released today a new version of Microsoft Worldwide telescope Autumnal Equinox Beta
Feng Zhao (Principal Researcher) talked on the energy usage, how to sense and how we can reduce. He showed a small device made by Microsoft Research that uses a 16-bit CPU, 10K RAM, 40K ROM to collect humidity, temperature and other parameters. It then transmits it using a low-power Radio as they are powered by batteries which need to last long.
He showed the below demo on how this data can be visualized:
This session was done very well by Eric Schmidt (Director, Microsoft).
Some top-level highlights were:
34 top level events sorted by icons
At the peak, 17 live separate events were happening
2000 hours of live content and 2200 Highlight content that was created
The result was staggering: 1.3 Billion Pageviews, 50 Million Unique Visitors, 70 million videos watched, 5000 Unique clips viewed per day during the final week, 600 million minutes of video delivery, 27 minutes of viewing per session, 35 million mobile views (external), 130,000 peak streams, 3.4 petabytes of video delivered, were built for 2.5 times of what was delivered.
Four main types of contents:
Live Content were delivered with Windows Media Server, with commentary was coming from commentators typing in a CMS which got moved as XML to production
Rewind – Video on Demand play of live content
Highlight NBC pulled out 50 Interns from college put them in 30 ROCK and make them create these 3-5min highlights of individual events.
Encores – Broadcast replays
Other points mentioned:
Planning of capacity was most important. When, Where and Size (each sport is of different length) was to be planned to determine the CPU, storage and ingress/egress needs.
NBC was helped by Intel Penguin processor, NBC waited for it and the servers got shipped around in May and took 6 weeks to go to Beijing
Bandwidth out of Beijing was limited to a 40 meg (DS3). 40 encoders were running live, so about 1MB per encoder (Digital Rapid). All this went into two windows media services box in Beijing, this got patched to window media services in 30 Rock, New York so that they can control if they had to. Which was then mapped to Limelight & Level 3 CDN’s massive WMS boxes. More details of encoding process in the blog post here
No full screen due to IOC Advertising requirements on the percentage of advertisements to video
Many partners were involved: Deltatre (Italy) had the CMS the best in the world to do live score on web
HTTPWatch Professional (and Fiddler) was useful to see what’s going on
Scrum and Scrumming builds better teams as the teams were distributed worldwide
Meeting Face to Face was very important, especially to bring this up cost in RFP stage
Everyone should know all roles and all architectural touch points
Reduce complexity via common schema
Long-tail delivery hides issues, when you are watching current items you needed to focus on the older contents that were being watched by the long-tail
The industry needs better telemetry and monitoring solutions
“Chunked” workflow (smaller sized thousand of files created) presented new challenges – Now IIS 7.0 Smooth Streaming in Media Pack announced yesterday does this better
Over 250 people between Microsoft, NBC and all other partners