root labs rdist

September 7, 2012

Toggl time-tracking service failures

Filed under: Security — Nate Lawson @ 10:33 am

A while ago, we investigated using various time-tracking services. Making this quick and easy for employees is helpful in a consulting company. Our experience with one service should serve as a cautionary note for web 2.0 companies that want to sell to businesses.

Time tracking is a service that seems both boring and easy to implement but is quite critical to most companies. Everyone from freelance writers to international manufacturers needs some system, ranging from paper and spreadsheets to sophisticated ERP systems. For a consulting company, lost time entries means lost money.

As we added employees last year, we began evaluating various web-based time tracking services in comparison to our home-grown system (flat text files).

We considered using Toggl, but our experiences with it started ok but got worse as they grew. We found that some test entries would disappear occasionally. The desktop widget was really an HTML app wrapped in a full browser (they started with Qt and moved to Chrome).

Then one day, we found that their servers had screwed up access control and everyone was being logged into a single account. You can see the messages in the attached screenshots. This was at once hilarious and horrifying, and we terminated our account immediately. No customer or sensitive data was lost, but there was no way we were ever going to use this service. There were various complaints about all this on the Toggl forums, but those posts were deleted when they updated their support website.

Dealing with distributed systems and the CAP theorem is particularly hard. Startups ignore this at your own (or your customers’) peril. Time tracking seems simple enough, but you can never lose data. For customers that don’t use code names for clients/projects, a data compromise could be devastating.

I debated whether to write this post since I don’t have a grudge against Toggl. However, I am hoping our experience will be informative as to what can happen when you outsource ownership of data that is directly tied to your revenue.

3 Comments

  1. Hello, thank you for bringing the case up. The incident you mention happened on March 9th, where one user account was partially compromised (using only our Nano and Classic timers) for about 2.5 hours. We had just upgraded our Ruby on Rails to 3.1, a version that changed some aspects of how HTML pages are cached. The problem was localized and fixed in a matter of few hours. It still is one of the worst cases we have had over the years, and certainly there were lessons that we took from it.

    I personally believe that cloud based applications are a lot more secure and reliable than most of the self-hosted servers. Sites are constantly upgraded, monitored and developed. Infrastructure is clustered, failovered and mirrored, data is constantly backed up – something that in-house servers usually lack.

    Comment by Alari Aho, Toggl CEO — September 10, 2012 @ 3:30 am

  2. It’s lucky that no personal information was compromised… it seems to me like that situation could have been worse. Did you decide on a different time-tracking system? We are currently trying out RescueTime.

    Comment by Sonja — September 25, 2012 @ 11:46 am

    • Sure, none of our information was compromised. Due to the above security issue and ongoing reliability issues (loss of entries), I recommend any other service or, better, keep your data in-house.

      Comment by Nate Lawson — October 12, 2012 @ 10:10 am


RSS feed for comments on this post.

The Rubric Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 81 other followers