Announcing MSFast By Yadid Ramot, Performance Architect, MySpace

Tracking User Experience

An overview of MySpace’s new open-source performance tracking tool - MSFast

We wanted to share with everyone an open-source project we’ve been working on at MySpace that, while relevant for developers, has a broader application for everyone. As developers ourselves, like many of you we are constantly trying to figure out the best way to profile and track the performance of our pages. After deep market research, we realized that the market for profiling tools lack features and capabilities which we think are important for efficient page optimization. These conclusions pushed us to develop a new homebrew tool, MSFast, which will hopefully cover all of these missing pieces.
It seems to us that the average web developer gives too much attention to tracking the time it takes for his content to download, and less-to-no attention to the order of the assets and the behavior of the page as it loads.

Tracking Downloads

The first problem you can have when tracking download speed is that the data is not normalized and could be inaccurate for long term trending. There are too many possible points of failure when tracking your site periodically, such as; your ISP changes your test box’s connection speed unknowingly (repairs/mistakes etc.), added a new hardware that change the bandwidth, installed a new local firewall that slows the connection, etc. These are just examples of local changes that will cause your site to appear slower/faster, but in the real world outside of your test box, these changes won’t be reflected.
Another problem you can have when tracking downloads is, while it’s important to keep in mind the simple equation of [slow connection + heavy content = slow site], there’s nothing much you can do about it to help speed up your site. So, you compressed your content and headers to the limit, removed the cookies and picked the fastest backboned CDN for your site, but on the user’s end, the site still act slow. What’s next?

User Experience

There are a lot of factors that need to be considered when optimizing the user experience on a page, such as ordering and prioritizing downloads, using the right type of download mechanism (blocking vs. async), utilizing the sockets correctly and that’s only for optimizing the content delivery. There are a lot of other elements that could cause bad user experience that has nothing to do with the actual download speed of the page. Elements with a really heavy JavaScript functionality, flash objects with a large amount of sprites that spike the CPU, heavy ActiveX controls that use a lot of memory, weird browser rendering quirks… these and more, can really clog the client’s machine and create a bad user experience when browsing our site.

Page Performance

As far as we know, the current common methods for profiling and capturing page performance are by:

• Running an HTTP sniffer when requesting the page and measuring each request
• Timing the page’s ONLOAD event
• Injecting a code before and after JavaScript calls and timing them

For the download side, using the right HTTP tracking software that generates accurate waterfall graphs can really help developers to re-prioritize requests and optimize the way content delivers by showing the order of downloads and which request blocked the pages vs. which were download in parallel.
On the events side, the ONLOAD and other before/after benchmarks can help developers measure specific and known functionality that blocked the rendering of the page.

What’s Missing?

Using the current available tools, we can track real-time download bottlenecks, JavaScript bottlenecks (as long as we know roughly where they are) and the ONLOAD event. Here’s what we felt was still missing:

• Normalize the download speed to generate accurate results
• Automatically detect performance bottlenecks caused by an object that spikes the CPU or clogs the memory
• A good way to tell when content is displayed and how long it took to render each UI element on the page
• Detect blocking browser rendering quirks

So to sum it all up, for our purposes, the ultimate performance profiling and user experience tracking tool should have:

• Normalize download data – Throttle down the inbound connection to ensure steady speed
• An HTTP sniffer – Track download speed and blocking requests
• JavaScript profiling capabilities – Profile how long it take to execute each function and detect blocking functions
• CPU and Memory profiling capabilities – Detect how much memory and CPU the page consumes and how much each element costs us
• Screenshot capturing abilities – Capture the page as it render to see exactly what was rendered during each event
• A mechanism that tracks the time it takes to render each segment of the page

During the past ~6 months, we at MySpace have been designing and developing a tool that would do exactly that (and more ;] ) and we ended up with “MSFast” - a browser plug-in that gives developers a complete, deep look at everything we could think of, from the moment the page leaves the web-server to the moment the page is completely rendered.

MSFast

In short, the way MSFast works is by setting up a custom proxy server on the client’s machine and using that proxy to 1. Normalize the download speed by throttling inbound data 2. Injecting the page with a set of JavaScript functions that executes a series of tests and measurements.

MSFast main features are:

Measure the CPU and Memory Consumption

MSFast captures the CPU and memory state of the browser’s process from the moment the page leaves the webserver to the moment the page is completely rendered.

Screenshots of the page

MSFast automatically takes screenshots of the page while it renders. These screenshots could then be reviewed against the rest of the collected data to see the actual UI state of the page.

Review downloaded files and show download time estimation on different bandwidths

MSFast captures the page’s HTTP traffic and stores the connection times. Since we throttle the connection and always ensure a constant tracking speed, we can display download estimates for each file, under different bandwidths.

Measure and show estimates of the time it takes to render each section of the page in different connection speeds

When the page loads, MSFast breaks it apart to roughly ~60 sections with an equal amount of HTML characters in them, and measures the time it takes to render each one of these sections. A developer can then review each part of HTML and see the time it will take to render it using different connection speeds.

Review the rendered HTML

Using MSFast, a developer can skip over the rendered code and see which section of HTML was processed by the browser during the page’s lifecycle.

Validate the content of your page against a set of proven “best practice” rules of web development

MSFast has a custom list of “validators” that could run against the tested page and alert the developer if their code goes against a set of common “best practices”. These rules are fully customizable and MSFast exposes an API for writing additional business-specific custom rules.

Open Source

We are dedicated to working with the developer community and see a great opportunity in giving back to the development community by open sourcing MSFast. We hope that this project will be one of the first of many future projects that we will open source ?

For more details and download directions please follow this link – http://www.msfast.com

By Yadid Ramot, Performance Architect, MySpace

Very shortly we'll be releasing a feature we are very excited about. Soon, app activities will be visible to friends of app users, regardless if they have the app installed. Though app activities have been useful in keeping existing users engaged, this has limited the viral utility of the feature.

As part of this feature we've included a few convenient and contextual options to each app activity. Activities for apps that a user already have installed will include in the activity stream an option to toggle the "receive activities" permission for that app. Activities for apps a user doesn't have installed will include a "Block app" option as well as an "Add app" option. Initially this UI will be released as standard links to inline modals. Shortly thereafter the UI will be updated based on a single click drop down menu system.  In addition, we'll also be adding another avenue for users to add your app by adding an "Add app" option to activities that appear within the activity stream module on a user's profile as well.

New App Activities for All Users and Links 

As always, the user experience is of utmost concern to us. We'll be closely monitoring the use of this feature on release. Any apps that are found to be abusing this functionality to the detriment of the user or in violation of the Terms of Usage will be dealt with immediately.

The team has put in a lot of hard work into this feature and the required infrastructure to support it. We hope that you enjoy this feature and it helps in the success of your app!

As mentioned in a previous blog, you are currently limited to sending 10 recipients per app invite. Starting in the next few weeks we'll be beginning a trial run which will double the limit to 20 recipients per invite. This trial will last for two weeks at which time we'll closely be monitoring the effects for both developers and the end user experience. From the end user perspective, the recipient field will now auto scroll if more than 2 rows of invitees are passed in.

Please keep in mind that during this time all existing limitations related to app invites will still remain in effect. This includes the moratorium on app invite looping. We look forward to seeing the results.

On Saturday May 30 in San Francsico, we announced the MySpaceID WordPress Plugin at WordCamp SF 2009.  Interest was so overwhelming that we ran out of flyers to hand out!

For those people who want more information, the plugin is available at:
http://wordpress.org/extend/plugins/wp-myspaceid/

Documentation on how to install it can be found at:
http://wiki.developer.myspace.com/index.php?title=MySpaceID_WordPress_Plug-in_Setup

What is the MySpaceID WordPress plugin?

If you have a self-hosted WordPress blog, the MySpaceID WordPress plugin lets your users log in with their MySpace credentials to leave comments.  This brings a few advantages:

1. Rather than leave comments anonymously, your users can now easily log in with their MySpace credentials in as little as 2 clicks and have their MySpace user name, picture and link automatically show up in the comment. Many users care about letting others find out more about them, and this is a really convenient way to do that, while boosting social engagement on your site.
2. After a user logs in, a cool-looking but unobtrusive bar appears across the bottom of the page. It lets the user share your blog with his friends simply by updating his status on MySpace.
3. The above status update contains a link to your blog, so that it drives traffic to your site (free marketing!), plus improves your PageRank.

How awesome is that?  In addition, the following features are coming soon:

1. When a user posts a comment, he can optionally share that with his friends as an activity on MySpace.
2. When you post a new blog entry, you can share it with your friends as an activity on MySpace.

This is the first release of this plugin, and as it undergoes several iterations in the coming months, we want to improve it with your input. Please make sure to post your suggestions and comments to our MySpaceID forum at http://developer.myspace.com/Community/forums/60.aspx.

We are really excited to be working with the WordPress community on a plugin that is a win-win for all.  Currently, our plugin works only with self-hosted WordPress installations.  If you have such an installation, make sure you install it and try it out!

Thanks to our developers who built this plugin: Chris Baker (www.myspace.com/christopherbbaker) and Steve Ng (www.myspace.com/stevenng, Twitter: @stevenng)!

Greetings Developers, 

The first batch of MySpace Open Platform DevJam videos has been uploaded!
Please click on a link below to view the video, and also check out our DevJam page for the uploaded presentations.

MySpace Open Platform - State of the Union
Presented by:
Jason Oberfest, SVP of Business Development for MySpace
Gerardo Capiel, VP of Product for the MySpace Open Platform
Max Engel, Product Manager for MySpaceID and the MySpace Open Platform

Building a Business on MySpace
Presented by Dan Yue, Co-Founder of Playdom
http:/www.playdom.com

How to Port Apps to MySpace
Presented Sebastien de Halleux, Chief Operating Officer and Co-Founder of Playfish
http://www.playfish.com

A Case Study on Monetization
Presented by:  A.T., EVP, Sales & Business Development of Zynga
http://www.zynga.com

Thanks to all who attended DevJam. It was great meeting with you. 

We'll be uploading more videos as they become available. 

Thanks, 

Rhonda

 

Greetings Developers,
 
DevJam starts today at 10:00 AM PST!
 
For those developers that aren’t able to attend the devJam in person, you can watch it live! Tune in to http://www.myspace.com/devjam2009 for all the details.

With this, we are excited to announce UStream’s implementation of MySpaceID – props to the team for an awesome implementation!
 
You can interact with fellow developers in real time as you watch the devJam. Chats will also appear as status updates back on MySpace and you can check out the profiles of other developers watching the sessions.  I encourage all of you to check out the devJam and this exciting new implementation of MySpaceID – It’s a great example of what MySpaceID can do and hints at what’s to come.

Also, we have another exciting announcement: Zynga will be presenting a case study on monetization.

You won't want to miss this!

Sign in, watch the devJam, and join the conversation! 

Coming in person? We can't wait to meet you. Hope to see you there or see you online.

Rhonda

View the full agenda here: http://myspacesfdevjam.eventbrite.com/

Greetings Developers,

A quick heads up, we need to take care of some routine maintenance tomorrow tonight. During this time, apps may be disabled and the developer site may be inaccessible. his planned outage starts at aprox 9:00PM tonight (PST) and will last aprox 2-3 hours. Expect downtime between 9:00PM - 12:00AM (PST).

Let me know if you have any questions.

Thanks!

Rhonda

 

We're happy to announce that we've just released the final piece of app notifications. You can now send app notifications directly from an OAuth REST end point. The REST end point allows you to pass in up to one thousand (1000) recipients at a time making it ideal for mass messaging communications to users of your app. This method also allows you to send notifications on a true server-to-server basis which is not initiated by a user action.

The general notifications page can be found in the developer Wiki. Documentation for the REST end point can be found here.

Enjoy!

Greetings Developers!

The MySpace Open Platform would like to invite you to another special DevJam Event.

This DevJam covers everything you’ll need to know about how to build great apps using  MySpace Open Platform. We’ll begin with the State of the Platform, and tell you all about what’s happening now and exciting things to come. We’ll cover everything from porting apps to MySpace, leveraging our messaging channels, making existing offsite applications more social with MySpaceID,  and everything in between.  You won’t want to miss this DevJam event!

If you can’t attend in person, don’t worry! We plan to livestream the event, stay tuned on that front for more details…

Guest Speakers include:

• Jason Oberfest, General Manager of the MySpace Open Platform and SVP of Business Development
• Gerardo Capiel, VP of Product for the MySpace Open Platform
• Dan Yue, Co-Founder of Playdom
• Sebastien de Halleux, Chief Operating Officer and Co-Founder of Playfish
• Max Newbould, Product Lead for MySpace Open Platform’s OpenSocial container
• Max Engel, Product Lead for MySpaceID

And many more!

Space is limited, so click below to see the full agenda and to register now!

http://myspacesfdevjam.eventbrite.com/

 

See you there!! :) :)

 

Rhonda

 

Just a quick heads up, sending notifications from the OpenSocial container is now live and ready for use. As I mentioned in my previous blog, this is ideal for implementation of action prompting messages, such as turn based indicators. Per the updated Terms of Usage, notifications sent from the OpenSocial container are limited to ten at a time, and all ten must be friends of the current app viewer. Please make sure you familiarize yourself with the full updated details in the documentation.

You can test out notifications in action by checking out a Simple Notifications App and a Advanced Notifications App.

Here are some current known issues:

• The App Notifications home page indicator isn't currently on. Expect this functionality to be turned on shortly.
• Notifications aren't currently deleted when a user uninstalls an app. Expect this shortly as well.
• Direct REST access to notifications (for mass messaging to all users). Just around the corner... ;)

Please let us know if you run into any problems or if you have any feature suggestions. We hope you enjoy this feature!

Many social web sites, Hi5, Orkut, MySpace, and others, all implement the OpenSocial specification. A number of the features in the specification allow you to get sets of items in one JavaScript call. These methods allow you to retrieve sets of friends, activities, and more. The OpenSocial specification requires the containers to support page sizes of 1 to 20 items per fetch. The spec allows for containers to support larger pages at the container's discretion. When working with a container, you may want to know what the upper limit actually is. You can find this information by searching the documentation of each OpenSocial container or you can write a tool that just tells you what the value is. Let's write that tool!

To find these limits, you will normally need access to an account with a moderately large number of friends, activities, albums, pictures, etc.-- about 200 of the value you are testing should work. The key part here is 'access' to such an account. In all but the friends case, you should be able to create an account with those characteristics without external assistance. For the friends case, you'll need to figure out which of your friends is well connected, then convince them to run this application and report back any results. Let's build an application that tests the friends case.

To start with, we'll set up a simple UI that allows us to state how far along we are in the testing. When all is done, we want to report back the maximum page size for this container. Our markup is:

Now, we need to write some OpenSocial code to discover those limits. The function we are writing will report back the largest page size that can be used for the current account. If you try this with an account that has 2 friends, the largest page that makes sense is 2 people. Ideally, you will have access to an account that has more friends (or other resource) than what fit in a single request. The idea is simple-- we start by testing a page of size 20 (the smallest 'big' page according to the OpenSocial specification). If that request succeeds and returns 20 friends, we double the number to 40 and try again. Eventually, we will either get all the friends in one request or we will request a page that is larger than what the container supports. If the requested size page is too big, the size of the resulting array will be smaller and will tell us what the maximum is.

We start by keeping track of the size of the last successful test in case we overshoot the top and start getting errors. We also have a simple output function that adds strings to the testOutput <div> tag.

We then need to run the tests. This involves initializing the outputs and then calling the workhorse function, findMaxPageSize.

findMaxPageSize takes a single argument: the maximum page size. I start out at 20 since that is the lower limit for any container. findMaxPageSize will have to follow the basic steps of getting a list of people:

1. Construct an IdSpec indicating who to pick and then which group. We'll get the viewer's friends.
2. Set the fetch params to indicate we want the first N friends by setting the opensocial.DataRequest.PeopleRequestFields.MAX param to N and opensocial.DataRequest.PeopleRequestFields.FIRST to 1.
3. Make the request and wait for the response.

In the response, we then:

1. Check if an error occurred. If so, reduce the size of the page and try again.
2. If no error occurred, get the list back. If the list size is smaller than what we asked for, we've hit a maximum for this person. Note: this is only absolute for the container if more friends exist.
3. Otherwise, try again with a larger number.

Here that is, reduced to code:

When running this on MySpace, I found that the maximum number of friends/page is 100 (as of today). This code works on OpenSocial 0.8 and 0.9 containers and can be useful for optimizing your applications for the different platforms you use.

With the initial release of App Notifications just around the corner I wanted to clarify some features and best practices for implementing app to user communications.

The first piece of App Notifications we’ll be releasing will allow you to send notifications directly from the OpenSocial 0.8 container.  This is the ideal method for sending “action prompting” messages to the user from the app, such as “Your Turn” indicators.  Sending notifications from the container will ultimately be limited to ten (10) recipients and they must be friends of the User.

The next component (which will be released shortly after the container functionality) will allow developers to send notifications directly from a REST end point.  Using the REST end point, developers can send notifications to any of their app users, making it ideal for digest or informational type messages. You’ll be limited to passing a maximum of one thousand (1000) recipients per call to this end point and it will be throttled (for obvious infrastructure reasons).

As part of our commitment to the MySpace App user experience, we’ll be making some specific updates to the Terms of Use. In addition to the standard rules of content for app communications (no advertising, no incentivizing, etc.), App Notifications obviously need to be directly and clearly related to the sending app. Any users you mention in notifications must be aware that their information is being included and you must provide a way for them to opt out. Please be sure to familiarize yourself with the new terms.

The entire team has worked very hard to get this functionality out and we’re excited to see the great new apps the community creates using it!

Sharing and testing apps in progress has historically been limited to just fellow app developers. This has made testing apps on any large scale prior to release difficult.  We’re happy to announce that we’ve updated the developer site to easily let you manage and add who has development and testing access to your apps.

From the My Apps section of the developer site you can now click the “Developers & Testers” link to view a paged list of your friends along with a “Developers” and “Test Users” column.

Simply drag and drop any of your friends into the “Test Users” column and they’ll have immediate access to view and use your app in progress.

Any of your friends that are registered developers (indicated by an asterisks [“*”] next to their display name) can be dragged into the Developers column where they will have full rights to your app source code.

Greetings Developers,

Just a quick note, Editor's Pick Category is LIVE!

We've gotten some great apps submitted to Developer Relations and have selected a few to spotlight here.  This list will be periodically rotating as we find more great apps to spotlight.

Thanks again for your participation and we look forward to seeing some more great apps!

Rhonda

As a developer, you’re currently limited to selecting only one recipient per app invite. For a user to send app invites to multiple friends has required developers to force the user to loop through the app invite process for each selected recipient.  Not ideal for a number of reasons for either the developer or user, especially for large recipient lists.

We’re happy to announce that we’ve now released native support for multiple app invites. As a developer you can now pass in an array of up to 10 recipients. From the end user’s perspective we’ve made a number of UI enhancements as well.  The user can now add or delete recipients directly from the invite modal using the standard multi-select component used in MySpace mail. This tool will auto recommend friends as users type. Best of all, it automatically filters out invalid recipients on the fly (eg. users who are not receiving app communications or already have the app installed).

Multiple Recipients

Once submitted, the user will receive a quick success confirmation that will automatically dismiss after a few seconds or list recipients whose invites could not be delivered in case of an error.

Since we’re providing native multiple app invite functionality and to preserve the app user experience we’ll be phasing out support for the existing looping invite process. We’ll shortly be updating our terms of service which will require the user to navigate away from the invite process page before allowing them to send another round of invites. We basically don’t want to encourage the same experience users have right now, but just on a larger scale. We’ll be providing a sixty (60) day grace period from the point the feature launches to fully phase out looping app invites for single recipients. In either case, you will not be allowed to loop multiple recipients.

Start implementing multiple recipient invites in your app now by checking out the docs.