What else can Spotify do?

Just read this article on huffington post on how hard spotify is trying to get new customers before the big boys (Apple, Google) enter:

While $400,000 is a significant amount of money to give to a potential competitor, Spotify is going where the eyeballs are, aggressively trying to court as many new users as possible before the already competitive streaming space -- Pandora, iHeartRadio and Last.fm are just a few of the established players -- becomes even more competitive. And that requires exposure, even if that means writing a check to a competitor.

The aggressive move seems inspired by a combination of fear and opportunity. For the moment, Spotify appears in an enviable position to capture a fatter slice of the emerging market for streaming music, but that moment may already be ending as enormous new competitors loom -- not the least Apple and Google.

It made me wonder what else can Spotify do other than spending a lot of money?  Any feature they introduce can possibly be copied by the big boys.   If it has a patent a work around could be found.  The other way to succeed is to use the Network Effect.  Spotify already is doing good things on the social front.  We have seen how ebay and craigslist are successful because of the Network Effect.​  By releasing more features that will be advantageous for a user because of his/her network on spotify,  spotify can stop users from jumping ship when the big boys come in.

I could think of 3 things that they can do now to gain a Network Effect

  1. The more people one brings/introduces/refers to spotify the more Karma ( a point system on spotify that can be but need not be public)​  points one gets. These points could be used for rewards.
  2. The more songs one shares with a friend (and the friends listens) the more good Karma  So if everyone brings their friends and keep sharing and listening to songs, more people are likely to be on spotify.
  3. Introduce spotify games that one can play with friends on spotify. Ex: A game where a user sends a snippet of a song he or she is hearing to a friend, and asks to name the song.

For a free user that reward could be some ad-free listening time.  For paid customers it could be some credit/discount to their service, or say a free month.​  Depending on the Karma points level one gets more benefits if they get to higher levels (similar to pyramid marketing) .  That could also add a gamification angle and we can compare friends scores (similar to foursquare).  It can also do ​others things such as users can get more karma points the more he or she listens.  In a sense spotify has to do price discrimination based on a user's network and create a path dependency so that user's will not switch out easily.

In one line, using the Network Effect to its benefit is the best defense ( in long term) against competition.

Web Development: A Crazy World

After leaving Raritan I've been in the process of retooling.  Used Java/Applets, Flash/Flex and now I know I have to move on to html5 and mobile.  While reading about all the possible tools/frameworks I have to learn, I came across the below rant on a discussion about this topic on Hacker News.  The below comment was very amusing/appropriate and to the point:

I agree, I can't keep up, I just finished learning backbone.js and now I've found out on HN that it's old news, and I should use ember.js, cross that, it has opinions, I should use Meteor, no, AngularJS, no, Tower.js (on node.js), and for html templates I need handlebars, no mustache, wait, DoT.js is better, hang on, why do I need an HTML parser inside the browser? isn't that what the browser for? so no HTML templates? ok, DOM snippets, fine, Web Components you say? W3C are in the game too? you mean write REGULAR JavaScript like the Google guys? yuck, oh, I just should write it with CofeeScript and it will look ok, not Coffee? Coco? LiveScript? DART? GWT? ok, let me just go back to Ruby on Rails, oh it doesn't scale? Grails? Groovy? Roo? too "Springy?" ok, what about node.js? doesn't scale either?? but I can write client side, server side and mongodb side code in the same language? (but does it have to be JavaScript?) ok, what about PHP, you say it's not really thread safe? they lie?? ok, let me go back to server coding, it's still Java right? no? Lisp? oh it's called Clojure? well, it has a Bridge / protocol buffers / thrift implementation so we can be language agnostic, so we can support our Haskell developers. Or just go with Scala/Lift/Play it's the BEST framework (Foresquare use it, so it has to be good). of course we won't do SOAP and will use only JSON RESTful services cause it's only for banks and Walmart, and god forbid to use a SQL database it will never scale I've had it, I'm going to outsource this project... they will probably use a wordpress template and copy paste jQuery to get me the same exact result without the headache and in halfquarter the price p.s. it would have been a longer rant today, I have lot of new things to add to it, sadly things didn't get any better: Now I'm trying to choose between yeoman and brunch, coffeescript vs livescript vs typescript, LESS vs Sass vs Scss vs stylus, Haml vs Jasmine vs that weird language called HTML, testacular vs mocha, fixtures vs mocks, RequireJS vs CommonJS, with almonds or without, I started using underscore then figured out it's already "old school" and I need to actually use lo-dash. So I think I'm going to take your advice ;).

Selenium with Sikuli

At Squarespace, I had to write automated tests for a css/js heavy website and that couldn't be done using just Selenium. Once I integrated Sikuli with Selenium, I was able to do get the job done. I had best of both worlds. I hosted a meetup at AOL/Huff Post office in NYC to talk about Sikuli and explain its benefits. The presentation can be found below:

There are 6 reasons to integrate selenium with sikuli. All about that is explained here.

Feedback received from the 40+ folks in the room, before and after the presentation:

This meetup was covered by huffington post here.​

Path To Test Automation Success

Back in 2006, I first came across a tool Selenium IDE that saved me a lot of time to do manual boring stuff. I slowly starting doing stuff that not just saved some time for me but also saved a lot of time for the whole team and improved the time to market for our products. That also great improved our output quality. Below is a brief presentation that shows the path I took. It doesn't get into a lot of technical details but is really a mix of process and tools.

How to Slow the Population Growth Rate

7 Billion and You
By now most of us know that our (earthlings) population has just reached 7 billion.  There are a number of discussions going on about the advantages of it and even more about the disadvantages.  With increased population we will have energy shortage, food shortage, water problems and many other problems.  Many people(countries like China) think that by controllng populaton by laws or by birth control education programs ( and some with incentives) we can slowdown the population growth.  I think that by educating people more, we can slow some population growth.

In rural parts of countries such as India, the children are looked as a help at in farmland.  By educating people, the farmland help incentive is gone and people will be producing less children.  In addition to that, educated people tend to get married little late and think about children even later. And by that time the fertility rate will come down and help reduce population.  Educating Women is another important factor.  The above-mentioned reasons apply to women as well and if we have two people with less fertility means even less children.

Urbanization is another imporant way in which population growth can be slowed.  In urban centres people have smaller places to live and that in-turn motivates the parents to produce less number of children.

In conclusion, instead of passing harsh laws(similar to what Sanjeev Gandhi did some years ago or what China does), I think focussing on other ways such as educating men and women is very advantageous.


How to Choose a Cloud Service Provider?

There are many articles and white-papers on this topic.  I just want to give a brief summary of thought process one has to go through before selecting a Cloud Service Provider.  Also, I will not be explaining what SaaS, PaaS and IaaS or Private Cloud here.  This is based on my hands-on experience, my analysis of the landscape and knowledge gained from numerous articles on the web.

First thing to decide is which cloud model to go for.  If we need to perform a non-core function such as payroll, CRM etc go for SaaS.  Leave that to the experts.  If we are doing an application that needs very high performance and scalability requirements then go for IaaS. Ex:- a payment processing system that has to respond in a second or a high traffic news website.   If your nature of business has to follow some regulatory requirements such as HIPAA, that data has to in Private Cloud(best option, though you could still find some IaaS to do it with a specific SLA).  
Now for the rest it really comes down to architecture of your application and if you have some code already present or not.  If it already has something working and it doesnt use any standards based coding then one has to pick IaaS.  If one is starting form scratch and wants to leverage the platforms developed by the PaaS providers they will choose PaaS.  Even though that will reduce time to market by a large amount, one has to be thinking about the lock-in. Sometimes one maybe locked into the platform.  Using PaaS will give some out of box features you do not have to write code for(like some social networking libraries or mobile platforms).  A new open source standard is just out called cloudfoundry. That will actually help PaaS to move accross vendors easily.  So how important is for one to move between service providers is the last important thing to think about while picking a cloud model.

Another important thing is to check their SLA for their uptime guarantee.  Providers like Amazon EC2 will even give you options of reliability.  If your company is new and not many customers are present, you can choose the cheaper one with slightly less uptime guarantee  

Another factor is the pricing model. Will you be charged per instance or per usage.  If you are a new company, you would prefer  a vendor that charges per usage(pay-as-you-go) and not per instance configured.  That will put the costs low and there would not be any commitment charges.

The next factor to look is Elastic Load balancing availability?  This becomes very important when your application becomes a hit and you have to scale for many users.  PaaS providers normally always have it. For IaaS providers one has to see available options and pricing models.

In addition to that, one has to see if the service provider is providing tools to monitor the health and also be able to control/diagnose them. Some vendors have automation features and they are very useful. Ex:- If CPU utilization % is greater than some configured value, you could be able to fire up another instance etc.  Some vendors provide only tools to monitor and configure but not diagnose in case of issues.  Some even have mobile apps to control and manage the services on the road.  We have to understand our needs before checking that factor. 

One need to be able to pick a vendor from which you can always migrate to a different vendor very easily.  One can also have two instances running on two different vendors and one can act as a spare. So if anything happens to one vendor resulting in downtime for the application,  shifting to other vendor is very easy. Spare becomes relevant only if we choose a not so popular/reliable vendors to save some money.

Some Vendors provide Linux or Windows instances and some provide you direct virtual machine instances.  If it is direct virtual machines you might have more flexibility in choosing your application's building blocks. In addition to that, you also have more freedom in moving between vendors is straightforward. All one has to do is pick a vendor in the list of vCloud partners.

Last but not the least  is Security.  One can ask the vendor a list of questions such as, who has access to servers, database and what are their disaster recovery procedures etc.  If a vendor is not ready to provide the security and access logs, then their overall quality becomes questionable.

I hope that for someone looking to pick a cloud vendor for their business or startup this write-up will set the mindset in the right direction.

More Speed or More Cores?

This is a common question for everyone about to buy new computer hardware.  From a regular Joe to a multi national corporation, everyone has to think about it.  I would like to give my take on this topic from both point of views and also on why this topic is more relevant than ever.

Nowadays we have very fast processors clocked most commonly at 2.2-2.4 GHz range but can even go to 3.46 GHz (i7 extreme).  With turbo boost you might even reach  3.73GHz.  The math is simple here. The more faster your CPU clock runs, the more instructions it can run and hence your video is converted much faster.  Same is the case with your mp3 conversion or any other operation one is trying to do.   But that generates more heat and consumes more power which is very bad for computers in general and laptops in specific.   If this is happening in a data center, the cooling costs increase by even large amount (As the relation between temperature increase and energy needed to cool it is not linear).

The advent of the multi-core processors, solves just that issue.  That is one can process data more faster without increasing the clock speed.  However a dual core 1.5GHz is not equal to a single core 3GHz.  Only of the operations are done in parallel one can achieve better performance using multi core processor. By having dual core some sets of operations can be done faster. i.e. If they are very independent tasks and the data fits in the L1 cache and do not need to load L2 and L3 cache from memory very frequently.  This will happen only if the applications and the operating system are designed with that in mind.  ex: An audio encoding application might have two threads, one reading the data in format 1 from a CD and the second thread is converting in memory data form format 1 to format 2.

Not all applications are designed to suit multi-core as doing that is more complicated and hence is costly.  With apple's Grand Central Dispatch (GCD) some of that responsibility is being shifted from the application to the OS.  That is a welcome change.  Applications that make use of GCD perform better better.  I hope other platforms do something like that soon.

Today this more speed vs more cores discussion is even more relevant due to the advent of smartphones and mobile devices. The iPhones, iPads  and the numerous androids out there are doing a lot more processing that mobile phones 3 years ago.  If they have to perform much better they need more CPU speed. But of that is increased too much, the battery life will go down and the device gets too hot.  Recently devices like iPad 2 and Motorola XOOM started having dual cores.  It is a welcome change.  I am not sure how many applications currently make use of the dual cores but I can see that one day 'hopefully' soon everyone will make applications with that in mind.

So by now it must be clear that all we need is more cores and just we have to make sure that the OS and the applications are made to use them.  When someone is building their own data center they would have already taken care of it.  A normal consumer just has to pick the OS that has built in support for that.
This is yet another way to Go Green!

So what will we have future? Maybe in the next 5 years or so we will have a multi-processor system (not just a multi-core) and the OS (ex:- Haiku) would be able to handle more than one processor very easily.


Please leave a line if this helps!

Cloud Computing and Entrepreneurship

By now almost anyone reading this blog would have heard about cloud computing.  Cloud Computing enables a user consume computation service without knowing its exact location.  Cloud computing is divided into 3 main categories:  'PaaS(Platform as a Service)' , 'SaaS(Software as a Service)' and 'IaaS(Infrastructure as a Service)'.  I will explain each type as and when needed.  


According to The National Institute of Standards and Technology (NIST): 
"Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction." 


It has a lot advantages including scalability, agility and cost effectiveness.  There are a number blogs about what it is and what its advantages are and I would look to focus on how it promotes Entrepreneurship.  Entrepreneurship can be further divided to tech startups, small businesses, performance arts etc.  I will try to explain how cloud computing can be used to start/succeed in each field.


Technology Startups: Lets say someone has a cool idea about the next facebook or google.  They would write a web application and then host it.  If it becomes popular then they can start charging money or use ads.  To launch a website and make sure it is available all the time one has to provide redundant servers and power.  Policies for backups and load balancing strategy should also be in place.  And many times the servers has to be residing in multiple location to provide reliable service.  For a 1-5 person startup all these maintenance and administration activities drain a lot of energy and money.  They could also use some hosting services that provide the above mentioned services.  Many times it comes with a high cost and some other contractual commitments.  For someone studying in a college or from a developing nation these costs are inhibiting factor.  This is exactly where cloud computing helps.  One can use any of the 'Platform as a Service' provided by the vendors.  By platform it means raw computing power, storage and I/O bandwidth.  Some main vendors in this field are Google App Engine and force.com.  


Once we use a platform as a service from any vendor they would take care of all the redundancy and backups and load balancing.  They would make sure that the application/website is up and running at all times and provide us with monitoring tools.  What is even better is that they provide you some free usage.  Google provides free 1GBytes of incoming and outgoing bandwidth and 1GB storage.  So until your website becomes famous you are pretty much not paying anything to google.  Lets say you wrote a web application that will show the results of some lottery that takes only once every month.  And your website has a lot of hits only on that day, you can just pay google for that day and the rest will be free.  It is pretty much like our electricity bill.  You pay only what you use.


Some platforms such as force.com will provide you additional features like social networking integration, drag and drop UI development,  mobile support etc.  These guys give one app for free and more than one app they will ask you to pay.  They also have app stores that already have a number of customer base.  So you can try to target then and start selling to them or let them try out for free initially.  So using these platforms you have access to potential clients.  Using platform as a service will make the tech startup entrepreneurs focus on their application and not the other maintenance stuff.


Small businesses:  Many businesses would like to have an online presence and maybe have an online order or status feature.  They would also like to have inventory management systems and also the ability to create invoices and manage expenses.  Providing support to the customers via phone, IM, twitter etc would also be a nice feature that would help them.  Normally small businesses would buy software from other and pay some consultants to make websites and get it hosted at some hosting service provider.  These kind of services would help them get more revenue.  But only by having more revenue one would be able to pay for these services.  Its like a chicken and egg problem.  Every small business owner would be facing this dilemma at some point in their life.  Cloud computing comes to the rescue again here.  These people would use something called 'Software as a Service(SaaS)'.  In SaaS you are rented an application.  You use an online application to fulfill your needs and you are charged based on your usage.



One can use website building and hosting services from the likes of wix or siteforce (many other alternatives available).  They come with different types of payment models.  Wix is free if you use their name on your site and agree to let them place their ads.  If you want to remove their name you have to pay about 10$ per month.  But they will help you to create and host your website and also provide you ecommerce solutions.  One need not hire a consultant to do basic activities.  So a small business can start with their own site initially and then once they make enough money they can use professional services to improve.  This way, chicken and egg is avoided.  For inventory management and invoicing one can use services from intuit etc.  They also have packages that are affordable to small businesses.(starting at 12 $ per month)  So one can just pay that as a utility and avoid high capital expenditure upfront.  For providing automated customer support one can use the tropo cloud.  They have some free usage and anything above that is pay as you go.  So if no ones the service no one pays.


Cloud has not only changed the way technology and business works. Now they started to change the way music is being made. Services such as Indaba, Soundation, Audiotool , Echonest could be used to collaborate, synthesize and release(on itunes store) audio albums over the cloud.  During Dreamforce 2010 will.i.am was also raving about using cloud to produce music collaboratively.  


There are many other miscellaneous cloud tools such as dropbox, soundcloud, skytap, iCloud etc that provide other services for consumers.  Among all the services I use dropbox and google app engine very often.  I have used force.com and wix.com but did not use the other services I have mentioned above. 


To conclude, cloud computing promotes entrepreneurship by providing below advantages:

  • No Capex (capital expenditure) and low Opex (operational expenditure) due to its pay-as-you-go model.
  • Worry about the your application or your business and not about the computing infrastructure
  • Scalability and high availability guaranteed.
One can say it is Outsourcing 2.0.  But it is not a passing fad as some critics says and it is here to stay.  Entrepreneurship is key for the global economy and Cloud Computing would enable that.  Cloud computing is also the most efficient way of computing.  This would help reduce overall energy consumption and hence reduce global warming.  
In short- Go for it.




Along Comes Windows 7 and the troubles too!

Along comes Windows 7 and the most prevalent question i hear is: How many drivers are not working on it? How many applications have problem running on it. Well the regular questions like, how long does it take to boot and Does it hang as often as Vista are still the most often asked questions. I was more interested in the first two questions.

Many people bought/sold applications which used to run on previous installments of Windows. Those applications might not be working on Windows 7. If it is a very famous application, the developers will be working around the clock to fix it. What if someone ordered a custom application for their office need and it doesn't work any more? What if vendor asks new license fee to make it work for Windows 7? I get it those are a lot of questions. There is a not so well -know application called VMware ThinApp .  (They have a 60 day trial available at :http://www.vmware.com/products/thinapp/). They have a simple and elegant way to convert applications that run on old versions of windows to thin virtual applications and make them run on Windows 7.

The main use of this tool is to save the amount of time it takes to develop and test that application for Windows 7. This would be a very big deal in leading technology companies. For companies which ordered a custom app from a vendor, need not pay that company again to make that app run on Windows 7. Using this anyone can make that application run on Windows 7 and hence save some money too. Other advantage of this tool is that you can install applications on USB stick and then carry it along with you. Via this method one can run it on any machine.

I used this to convert the MG-Soft MIB Browser tool to a thin app. This tool (the particular version i had at hand) was not installing on Windows 7. In less than 10 minutes i was able to make it work on Windows 7. Anyone who needs some help in making applications run on Windows 7 should try this.
As usual please leave a comment if this helps.

The Other Uses of Virtualization

The other uses of Virtualization. Everyone agrees that virtualization is very useful for running corporate data centers as it not only reduce energy consumption but also helps scale the applications in an efficient way. Disaster recovery and High Availability are others reason everyone use virtualization. Furthermore cloud/distributed computing is best use of virtualization. This post is not about the above applications. There are a lot of posts and white papers on the web to talk about the applications of virtualization in the above areas. This post is about the other uses of virtualization : standardizing development environments, helping kick start new team members with their development environment , restoring our workspaces in case we mess it up and perhaps the best use is in scaling up our test automation and/or stress/load testing efforts just before major releases. Every project has a set of tools which needs to be installed and configured in a particular way before a developer starts contributing to the project. Many times it takes a day or two for a new hire to join a company and setup all the needed tools. In some projects all the steps are documented and in some other projects these steps are not documented. The new hires would just have to figure out by asking others or by just trying a couple of ways before arriving on the best practices for that project. As we all know, many developers do not like documenting and fail to do it many times :). If we use a virtual machine(VM) as part of our development environment and install and configure all our tools on the VM, it will save a lot of time and frustration for new team members. This will also ensure that everyone is following the best practices. All without even a single line of the dreaded documentation! To illustrate this lets take the example of a web development project. Many people use eclipse for doing web development. Every time a new plugin is released or some new frameworks are out the developers are eager to try them. During those times, they end up updating eclipse or doing some other environment changes. Sometimes this has side effects, i.e. out main project stops working :(. To revert all the changes they just did will again cost them some time. It happened to me also number of times. In case we use a virtual machine as our development environment, if anything goes wrong we will stop the current VM(well after checking in all the code needed) and then just boot the baseline copy virtual machine and we will be all set. If we are using ESXi we can take a snapshot and do our experiments. If anything bad happens in the experiments we can just go back to the snapshot.This logic can be applied to a large variety of projects ranging from embedded systems to enterprise applications. Another very good use of virtualization is in testing. Before every major release (we can extend the logic to every release ) we have run a lot of tests. If a company uses test automation and has tests which would take 7-8 days to run the entire suite then by using virtualization we can just create 7-8 copies of the testing virtual machine. Then run all the tests in parallel in just one day. After our tests are run, we can shutdown all the unnecessary testing VMs(virtual machines). Doing this would save us a lot of money i.e. both the cost of buying new hardware
and of maintaining it. For instance in one my recent project, we used selenium, testng and Java to do test automation. We had installed all the tools we needed to develop and run the test cases on a virtual machine. Our test suite currently takes approximately 24 hours to run. We used 3 virtual machines to run the tests overnight. This helped us save a lot of time.

Another use of virtualization is during stress testing, soak testing and load testing. To test any appliance/web appliance or a website, we need a lot of stress testing. Most of the times this is just one time thing, just before the main release. We try to simulate real life scenarios using a large number of simultaneous client connections to our server. For a bank website we probably have to simulate more than a couple of thousand request at any given point of time. We can use the power of virtualization to scale up our stress/load/soak testing. Once we are done, we can shutdown all those test virtual machines. In this economy reducing cost while doing thorough testing is definitely going to impress your boss.

As we are done with the uses, now i would like to talk about some virtualization tools i used and like. All of them are free and are by VMware. Googling the names will give a lot of tutorials so i am skipping that for now.

VMware Player: It runs on windows and linux. Using this we can run virtual machines on our desktop. i.e. We can boot in windows and then boot a Virtual Machine of Linux using VMware Player. Once upon a time everyone used dual boots with windows and linux, now we need not do that anymore(oh- thats yet another use of virtualization) In this case the windows OS has the full control of hardware and passes it to other OSes which are running in the VM.

VMware ESXi (vSphere Hypervisor) : This is a more efficient way to run VMs. If you need better performance this is the tool which needs to be installed. In this case the VMware OS(ESXi) has the full control of hardware and then passes it to the installed OS inside the virtual machine.

VMware Standalone Converter: This is used to convert the virtual machines one format to other. All the VMware tools do not use same format. This is used to convert VMs from one format to other.

VMware vCenter Converter: This is used to convert physical machines to virtual machines.(doesn't work with all the target operating system though)

VMware vSphere Client: This is used to connect to ESXi or vCenter.

Here is a link for the full set of tools from VMware: http://www.vmware.com/support/

VMware is the only vendor in virtualization, but it is the most popular one. Other VM major vendors are:
Citrix Systems XenServer: http://www.citrix.com/English/ps2/products/product.asp?contentID=683148
Sun Virtual Box : http://www.virtualbox.org/
Microsoft Hyper-V : http://www.microsoft.com/virtualization/en/us/default.aspx
Open Source xen: http://www.xen.org/


In summary, i think that the uses of virtualization are way beyond its current applications. Scaling and risk mitigation would still be the major uses but nevertheless
the other uses are also important. All the articles on the web just talk about how virtualization is useful for saving money for companies, its uses for a developer and tester have been ignored. Also using virtualization we can do things which we probably never did before. Ex:- Scale a load test to a very large number.


Bottom line: Virtualize, Scale and Go Green!


Disclosure: I currently own the stock of VMware and Citrix.