Software quality the main course of QCon
November 9, 2012 —
(Page 2 of 3)
Related Search Term(s): QCon
After building an interpreter to stand in for the compiler in development machines, the wait time was cut down to around 10 seconds, but for Mortensen and the developers at Facebook, this was still too slow. The answer was to use a JIT compiler.
“We turned this on this spring, and got some awesome results,” he said. “When we went to the JIT compiler, there was cheering in the streets. But as a reminder, it's never done. The site continues to get richer. This is one of those things that they continue to keep fighting on. It's been a fun saga. With enough effort, almost anything that matters can be made efficient. You really want to make your engineers fast.”
Another popular topic at QCon was the rise of new data stores. While many of the NoSQL and database companies were on hand, much of the discussion was around best practices for any data store.
Jeremy Edberg, manager of cloud operations and reliability engineering at Netflix, said he learned how to store data for quick access while he was working as an engineer at the news site Reddit.
Of the Reddit stack, he said, “There was Cassandra, Postgres and memcached. The biggest way Reddit does their resiliency is through the sharding of Postgres. Reddit splits writes across four master databases, which have different types of things. Each one has at least one replica in another zone. Votes on links only get one replica, while comments have 17 replicas spread across different zones. Reddit uses Postgres like a NoSQL store. There are just two tables: a thing table and a data table.”
Edberg said one of the keys to success was using consistent key hashing in memcached. “It's important. If you're using a naive hashing algorithm, which every memcached library uses by default, [you can run into performance issues],” he said. “If you add a server, it will shift three quarters of your data to a different server, so you invalidate three quarters of your cache. Consistent key hashing is the reverse: One quarter of your data will be in the wrong place. This makes it so you don't put a huge load on your database when you add a new node. Protip: Use consistent key hashing. If you're using memcached, you're probably using a naïve one if you haven't set it manually.”