Master Developers: The Ember.js Core Team
Single page apps are the new hotness; everyone's trying to figure out the easiest way to build them. But it's more than just finding a couple of controls to slap together and sprinkling Ajax pixie dust on it. Building scalable and maintainable apps is serious business, which requires serious tools.
Ember.js is a very serious framework for doing just that. Check out the interview I did with Ember.js Core Team leads, Yehuda Katz and Tom Dale, as they discuss what prompted them to begin the Ember project, its design philosophy, and where it fits into an already crowded library ecosystem.
Q Tell us about your professional backgrounds.
Yehuda: I was an Accounting major in college, with a whole bunch of interesting minors (Journalism, Philosophy, History, TV/Radio). I loved learning, but somehow missed the fact that Accounting as a profession was pretty boring, at least to me.
I had rejected a computer science major on the grounds that it was too hard and "not for me," even though I had done a small amount of QBasic programming as a kid and some Visual Basic in college. I thought that BASIC was a toy language, and not capable of doing real work, because I picked up a C book when I was a kid and found it impenetrable.
I got extremely lucky to see an internal job posted at my first place of employment for a web designer, and thought "I did print design in college, that's the same thing right?"
I was also aware of Knockout, but I wasn’t a huge fan of packing all of the binding information into HTML attributes.
I very quickly realized that I would need to learn real programming, and was lucky to be getting started just as jQuery and Ruby on Rails were getting off the ground. I quickly got involved in Open Source, and because those projects were still so young, was able to provide a lot of value even with my very limited programming skill.
I got involved with Merb, a Rails competitor, after my first real application quickly grew outside of the bounds of the Rails happy path and our small team of three found ourselves spending more time hacking around Rails than writing application code. To be fair, we also wrote very few (if any) tests, so maybe that had something to do with it. ;)
Eventually, the Merb team realized that our core ideas were quite similar to the ideas of the Rails team, just with a different focus. We joined forces to bring more modularity and configurability to Rails for advanced users, while carefully avoiding breaking the happy path that works so well for beginners. My work on Rails 3 was done almost exclusively pair-programming with Carl Lerche. That experience is what gave me an appreciation for the use of pair programming to think through hard problems and implement their solutions well.
Once we shipped Rails 3, I started looking at the web ecosystem to see what other problems existed. I was horrified to see that the state of building large web applications, both on desktop and mobile, hadn't changed a lot since I started with jQuery so many years before. For all of jQuery's strengths as an excellent DOM library, it had somehow trapped a generation of web developers into the same low-level abstractions. Something needed to be done.
I was also aware of Knockout, but I wasn't a huge fan of packing all of the binding information into HTML attributes. This was mostly an aesthetic concern, but sometimes aesthetics matter.
Around this time, Charles Jolley, the creator of SproutCore, learned that I was working on this project. He already knew me because the original version of SproutCore was built on Merb, and he had invited me to speak at Apple during the Rails 3 timeframe.
He suggested that we band together and that I bring my templating ideas to SproutCore, which already had a lot of the data binding infrastructure I was trying to hand-roll. Having seen this story before with Merb and Rails 3, I jumped at the chance to take advantage of a battle-tested data binding system to pair with my templating solution. I joined Charles at his new company, Strobe, to work on SproutCore.
At this point, Tom was still working at Apple on SproutCore, and we met a few times to discuss new APIs that Apple was adding to SproutCore. I quickly realized that Tom had very good API design chops and moved to recruit him to work at Strobe. When he joined us, I began pairing with him on SproutCore 1.6, the first version that would include the templating solution.
When Strobe was sold to Facebook, I formed Tilde with partners in crime Tom, Carl and Leah, to continue working on this project, outside of the confines of a VC-backed company. I've been there ever since.
Tom: Well, I'm certainly not a classically trained engineer. I picked up a book on BASIC from the public library when I was a kid, but we had a Mac at home and the development tools at the time were out of reach. So after a while, I kind of gave up on programming. It seemed too hard for me. I ended up getting a degree in Criminology, Law & Society from UC Irvine. I think the only real programming I did in college was slapping together a Ruby on Rails site for my World of Warcraft guild. (I can't believe I'm admitting this publicly.)
After I graduated from college, I was working at the Genius Bar of an Apple Store in Southern California. The software they gave us for managing the paperwork for repairs was quite painful - you know, the standard enterprise stuff you'd expect.
I decided that I could probably make our lives easier, so I picked up a book on Cocoa and started hacking on an app to automate a lot of the stuff we had to do by hand. After awhile, it started to spread virally to other stores in the region.
I had a buddy who had ditched retail and gone up to Cupertino to work for Apple corporate. He heard I had programming experience, and needed someone to work on a project that was using Ruby on Rails on the backend and SproutCore on the front-end.
Yehuda and I both felt like we needed to be independent to accomplish our goals in open source.
A few months later, we were close to shipping our first version. Since SproutCore had not yet reached 1.0, we were working closely with Charles Jolley and the rest of the SproutCore team, and I got to know them quite well.
Working on open source at Apple was, well, interesting. I discovered that I really loved working on frameworks that help other developers. By then, Charles had left Apple and started a company with Yehuda, called Strobe. I got to know Yehuda through our mutual work on designing new APIs for SproutCore. I started getting the feeling that I could have a bigger impact on open source, if I was able to work on it outside Apple. So, I left Apple to join Charles and Yehuda at Strobe.
Strobe was a fantastic experience and I learned a ton from both Yehuda and Charles. Eventually, we decided that SproutCore had a lot of great ideas, but too much legacy. We started from scratch on what was then called SproutCore 2.0.
Like many startups, Strobe was acquired by Facebook. While Facebook is an exciting company to work for, Yehuda and I both felt like we needed to be independent to accomplish our goals in open source. We, along with our two other co-founders Leah and Carl, started our current company, Tilde, at the end of 2011.
We primarily make our money consulting, which we use to pay for time to work on Ember.js and other open source projects. We're also working on some products that we think developers will love.
Q The million dollar question, "Why build another framework?"
So the question isn't, "Why build another framework?," since we were one of the first. ;) A better question is, why are there so many other frameworks out there now? And I think the answer is that a lot of this stuff looks a lot easier to build than it really is.
Take bindings, for example. Any competent engineer could build a simple binding system in a day. But it turns out there are a lot of edge cases, race conditions and infinite loops possible in larger systems. Getting something rock solid has taken a long time.
It gets even harder when you take all of these features and try to make them work together. No other framework is attempting to tackle the problem from top-to-bottom the way Ember.js is, while keeping it modular so that you can swap out different parts to make them work however you'd like. And developers using Ember really appreciate how much thought we've put into how each of these features works together; it feels designed, not slapped together.
Yehuda: The primary reason to get started on a new framework was a strong desire to see HTML and templating take center stage in a major framework. At the time, in a desire to be "agnostic", many of the solutions said, "You can use a template engine... or no template engine! Just get some DOM on the screen!"
I felt very strongly that a next-generation framework would be built around data-bound templates. Since then, Ember.js has grown a lot, and we focus as much on application structure and routing as data binding, but none of that would be possible without HTML that could update itself as the user navigated around.
Q I believe Ember.js came out of your work on SproutCore. What prompted the rename and new effort that's now Ember.js?
Tom: While SproutCore was ahead of its time, it made a lot of mistakes, as well. Perhaps the biggest mistake was trying to port the Cocoa UI to the web. Shipping with widgets is great if that's all you need; but especially on the web, customizing your UI is of the utmost importance. Doing that in SproutCore is beyond painful.
Backbone's popularity was a wake-up call for us. It proved that developers really wanted to build these incredibly responsive, client-side apps. But they also had years of experience crafting user interfaces with HTML and CSS, and we couldn't ask them to throw that away to learn SproutCore.
Backbone's popularity was a wake-up call for us.
Having been building apps like this for longer than just about anyone, we knew immediately that Backbone didn't have the right abstractions to help developers build large apps. But it was easy to get started with, and let developers use their existing HTML and CSS know-how.
While a lot of people think of SproutCore as just "native-like controls for the web," the reality is that it also embraced the architectural patterns of Cocoa that have made Mac and iOS apps so successful.
We knew we could bring those tools to web developers without the cruft of the SproutCore view layer, while making the API easier to get started with. Additionally, we wanted new users to be able to lean on their existing skills as much as possible.
So we started over, completely from scratch, on what, at the time, we called SproutCore 2.0. Unfortunately, the name SproutCore carries a lot of negative connotations, and the new project was suffering from that, despite not sharing a single line of code. Additionally, many members of the SproutCore community found our template-driven approach controversial. We decided that a split from the SproutCore community, as well as a rename to Ember.js, was a good way to send the message that the project was a new beginning.
The goal of Ember was to give developers back the tools they were used to using.
Yehuda: My biggest problem with SproutCore, as I was working on it, was the amount of time, energy, and code that was spent on abstracting the DOM. Frankly, the number of web developers that understand HTML and CSS dwarfs the number of developers willing and able to learn yet another abstraction on top. And when it came time to style or theme a SproutCore app (because who wanted the default theme), you'd find yourself delving deep into the arcane world of RenderDelegates and RenderContexts.
It was all for a good cause, but at the end of the day, the web platform presents an extremely powerful framework for presenting and laying out content via HTML and CSS. Is it perfect? No, definitely not, although it's getting better all the time. But it's universal and ubiquitous. The goal of Ember, from the beginning, was to give developers back the tools they were used to using instead of asking them to learn a whole new set of tools loosely derived from Cocoa.
Q Ember is still a baby in terms of frameworks. What have been the challenges of launching a new OSS effort and gaining traction?
Yehuda: Open Source projects look easy from the outside, but they're probably the ultimate chicken-and-egg problem. You need a successful project to bring in contributors and users, but those same users are the ones who will make your project successful in the first place.
the reward for dealing with the instability is a much more stable 1.0
The only real solution is to personally bootstrap the project by being the chicken and egg at the same time. You need to personally work with contributors and users at the same time as you build up the project to a reasonable degree of usability (and eventually, stability).
One thing I would say is that for all the pain that this may cause, it's critical for a new project to have a time for instability as you're still working out exactly what the goals of the project are. As you gain new users, many of them will provide valuable insights into problems you weren't even thinking about when you started. You must have the ability to change APIs to adapt to these learnings.
That said, you must also put a stake in the ground at some point and release a much more stable 1.0. As Tom and I often say, the reward for dealing with the instability is a much more stable 1.0, because many of the kinks have already been worked out. I would recommend a strong commitment to Semantic Versioning once you have gotten to 1.0.
Finally, you should nurture contributors who do things other than work on hard code problems. Some of our best contributors write documentation, maintain infrastructure, and help manage logistics for both Ember and community members looking to do meetups. To Ember, these contributors are just as valued as people who do deep performance work in Ember, and rewarding them will keep the community strong and healthy.
Tom: The web is an interesting place. You know, if you're Apple, you get to say, "Here's Cocoa. If you want to make money on the App Store, write your apps using this." There's a pot of gold at the end of the rainbow. People are willing to sit down and learn whatever you tell them to learn.
Anyone can decide to write a new framework or library and publish it instantly.
The web is different. On an open platform, anyone can decide to write a new framework or library and publish it instantly. There is an incredible amount of noise mixed in with the signal. As framework authors, we need to show value within five minutes, or the person checking you out will bail and check out a competitor. So the challenge for us is, not only do we need to bring the concepts of Cocoa to the web, we also have to make them easier to use! And in the beginning, we were only two people.
Thankfully, smart people are attracted to solving hard problems. I think some of the most talented web developers in the world recognized early on that we were going to tackle a hard problem, and we were going to do it right. Our community has grown explosively, and most of the work happening on the framework now is not from me or Yehuda. It's incredibly gratifying to see your message - that the web needs better tools, and we can build them together-resonate with other developers.
Q In terms of MVC, I've read that Ember.js takes a slightly different approach to the paradigm than other frameworks. Can you explain the differences and design choices?
Tom: Ironically, Ember.js is probably closest to the original MVC, as used in Smalltalk in the 70's. Since then, server-side MVC frameworks like Ruby on Rails have become very popular, and, in most developers' heads, subsumed the term.
Ember's MVC is just like Cocoa or Smalltalk. Models represent domain logic and data. Views (which are usually templates) are bound to a controller, which typically represents a model. Changes to the underlying model are propagated up to the view, automatically. Unlike in something like Rails or Django, where objects are re-created after each request, these objects are long-lived; usually for as long as the user has the application open in her browser.
In Ember, it's easier to write an app that works this way than not. We designed the entire API to center around the idea of URLs, so it's something you get from the beginning, instead of being something you tack on at the end. You can feel the difference.
Yehuda: I actually don't think there is any one MVC paradigm that other frameworks take. The main thing that everyone shares is a desire to decouple the HTML that ends up in the browser from the data models that power them.
Ember’s approach is inspired by Cocoa
Backbone, for example, stops there. You get Models and Views, and if you want other layers you can roll your own (and many do).
Other frameworks use the "controller" layer as something very close to views. These frameworks often use terminology like "View Model" to describe this layer. I believe that Knockout uses this approach.
Ember's approach is inspired by Cocoa, which uses controllers to decorate model objects for presentation in the view layer. Our router serves the same role as Cocoa's "coordinating controllers", making cross-controller coordination possible. Because Ember is a web framework, the router makes URLs a front-and-center concern, guaranteeing that when you build out your application's structure, you get bookmarkable and shareable URLs for free as a side-effect.
Q As I'm going through the learning process, I feel a lot of Rails influence in the Ember design. Am I off on this?
We realized that Ruby on Rails had long ago figured out how to orient a framework around URLs.
Tom: Like I said, the URL is the key feature of the web. We knew that Cocoa had the concepts necessary to build long-running, stateful applications. But we couldn't port the API literally to the web; both SproutCore and Cappuccino had tried this and failed.
When thinking about the problem, we realized that Ruby on Rails had long ago figured out how to orient a framework around URLs. In most Rails apps, models are just resources that you expose using conventional routes.
So, the Rails inspiration you feel in Ember.js is us pairing the architecture of native apps with the URL-centricity of Rails. And, like Rails, we also value convention over configuration!
Yehuda: Interestingly, when we started working on Ember, we spent a lot of time being careful not to cargo cult precise ways that Rails does certain things. Our position was that Rails-style MVC was different from Ember-style MVC, so copying any superficial similarities would probably do more harm than good.
The feeling of Rails similarity mostly comes from our strong adherence to convention over configuration, and particularly naming conventions. Our MVC may not look a lot like Rails', but the fact that you don't have to wire up objects together, instead relying on naming to do the wiring for you, is very inspired by Rails.
Ironically, after all of our desire not to copy Rails, it turned out that the Rails router was a very close fit for what we ended up needing in Ember, and the API (mostly coincidentally) ended up converging on something close to the approach used in Rails.
Q Along those lines, what are the key things that new developers to Ember.js should know about?
Tom: Templates are connected to controllers, which are themselves connected to a model (or a collection of models). These connections are set up in the router. Building large Ember.js applications is just repeating this pattern, over and over. Template, controller, model.
- Ember.js does a LOT for you and in some cases, it feels like black magic. I know a lot of developers don't like that. What "constructive feedback" (i.e.: don't let Tom answer) would you offer them which this sort of black boxing of code?
- Routing, I've found, is one of the most crucial parts of Ember. From a performance perspective, it would seem that hooking up all of these routes and controllers would impact performance but the small app I've built seems fine. What's the largest scale that Ember has been tested at?
One of the core principles of Ember that lets us get great performance is that we do everything as lazily as we can. We never pre-compute anything that we can do just-in-time. Laziness may be frustrating in humans, but it's a boon for web apps! The same thing is true of loading models, creating views, setting up bindings between views and controllers, etc. We spend a lot of time thinking about performance.
I'm not sure what the largest application out there is. A lot of businesses are betting big on Ember.js and building their next-generation web applications with the framework. That means that we don't get to see the source code for most Ember apps!
Yehuda: Ember has been used on some really big apps, like Zendesk, Square, Travis CI and Discourse. All of these apps make use of large amounts of data that are pushed through the Ember binding system.
Square, in particular, has done really amazing work combining Ember.js and Crossfilter to allow exploration of hundreds of thousands of data points without going back to the server to display the results.
Q Over the last year, the API has gone through numerous revisions. This puts high maintenance demands on projects that want to use the framework. Where are you guys at with locking down the API and how will you handle deprecating features in the future?
Tom: Our early adopters have been incredibly patient with us while we've been perfecting Ember. In fact, all of the API churn is from spending lots and lots of time with developers using early versions of the framework, and getting their feedback. We've been incorporating that feedback at a rapid clip.
The downside is that we've kept people on their toes staying up-to-date. The good news is that we have a much better product than other frameworks that locked down their APIs long ago. We're incredibly happy with where we've ended up, and we announced the Ember 1.0 RC at EmberCamp last week. We will be following the SemVer standard, which means that apps you build today will be compatible with the framework until we release a version 2.0. Which, for the sake of our sanity, hopefully won't be for quite a while!
Yehuda: The pattern I like to use in Open Source projects is to allow early adopters to drive the API before a project hits 1.0, and then lock it down tight once that milestone is reached. This gives early users the opportunity to provide feedback about use-cases that the original team may not have known about, and the ability to drive the direction of the framework. It also makes it clear that there is a ticking clock on that kind of exploration, and makes the 1.0 milestone meaningful.
Q The Discourse team just launched their site and made the use of Ember.js a key talking point. What was your involvement with that effort and in your opinion, what were the positives and negatives learned from that experience?
Tom: The Discourse guys have done just an incredible job. I am still stunned at what a polished product two engineers were able to build using a framework that was undergoing heavy development.
Jeff Atwood came and showed us an early version of Discourse late last year. Their plan is to build forum software for the next 10 years of the web. That lined up well with our goals - we want to build the framework for the next 10 years of the web.
We helped answer questions and give suggestions, but the two Discourse developers-Sam Saffron and Robin Ward-really are superlative developers. If anything, they really helped us benchmark Ember and gave our performance wizards-core team members Erik Bryn and Kris Selden-areas to focus on. The devotion to performance of these guys has really paid off for the end users of the framework.
Yehuda: One of my favorite things about Discourse is that it is an immersive, very responsive application that is still mostly displaying content. This is the kind of application that many naysayers say needs to be built using raw HTML, in order to ensure a good URL experience and showing up on Google.
What Discourse showed is that you can build a content site with rich interactions without giving up the URL-friendliness of static sites. And they show up on Google just fine!
I like to believe that the Ember router, which insists that Ember developers build their application flows in terms of URLs, helped to make this possible. In the coming months, we'll be looking at patterns that they used for things like infinite scroll to see whether we can roll some of those approaches back into Ember.
Q In terms of project team, you've purposely chosen to keep it small. Tell me about the decision behind that and how you feel the community can best contribute.
Tom: Yehuda and I have a very specific vision for the framework. It's important to us that contributors share that vision. I think one thing I've learned from Yehuda-that he's learned from his experience working on big open source projects like jQuery and Rails, and on the standards bodies-is that most developers like to solve for specific scenarios.
Yehuda and I have a very specific vision for the framework
It's extremely easy to tack on a feature that solves a particular problem someone is encountering. It takes a lot of effort to think through the underlying cause of that problem, try to think through all of the possible use cases, and come up with something that solves that specific problem as well as others, with the minimum number of new concepts.
People get impatient and even angry while you're still thinking about the best way to solve a problem. But the end result is worth it.
It's hard to find other developers who have the resolve to say "no," and not just rush something in because it fixes a particular problem. But, again, it's worth it.
Yehuda: We just announced our core team, which has eight members. Veteran Ember users will recognize all of their names, and there are a number of people who will probably join the core team this year if they keep up their current level of involvement. Rails, which has been around for much longer, has 14 core team members.
Tom: Ember is the only framework that both wants to solve the entire problem, top-to-bottom, and that also cares about crafting an API and documentation that is approachable and user-friendly.
Developers love to argue, so our strong conventions allow teams to avoid bickering over what we call trivial choices. For example, Ember expects templates to be named similarly to the controller it's connected to, which is named similarly to the model and route. It makes it easy for an Ember developer to jump into the middle of a new project and easily figure out what's going on. By comparison, every Backbone app that I've worked on has been like a completely different code base, since every team uses it a little bit differently.
Yehuda: Over the past year, one thing that we've really taken to heart is that if people are building web applications (as opposed to native applications), they need to make URLs the front-and-center way that they structure and organize their application.
Other frameworks provide support for URLs, but only Ember starts new developers with this crucial aspect of the web experience from day one.
Q What do you see as the key deciding factors for choosing something like Ember.js instead of using solely a library like jQuery or MooTools?
But neither give you any architectural tools.
Tom: jQuery, like MooTools, is a fantastic library for manipulating the DOM. But neither give you any architectural tools for doing things like fetching models, keeping the URL up-to-date with what's on screen, rendering templates, or managing application state. You can try to do it by hand, but my experience, and the experience of every developer I've talked to about this, is that you end up with messy spaghetti code that becomes impossible to decipher.
Yehuda: Ember is an application framework, while jQuery and MooTools are libraries to abstract away common ways of interacting with the browser's API. Ember actually uses jQuery under the hood to manage the DOM, so you can see that Ember builds on the DOM to help developers structure and organize their applications.
In my opinion, if someone is truly torn about whether they should use a low-level library like jQuery or an application framework, they should probably go with jQuery until they hit issues that would benefit from a framework.
Q I noticed that Ember uses jQuery. Can you tell us about this choice?
Tom: jQuery is just awesome. It has effectively become the standard library of the web. I love it because I don't have to worry about weird cross-browser bugs. A lot of people think that jQuery is less useful now that older IEs are fading out of existence; but those people are just wrong. WebKit and Firefox bugs are just as bad, and sometimes worse, than many IE bugs. The extremely pleasant API is just icing on the cake.
Q In terms of mobile, what do developers need to consider when using Ember?
Tom: Like any framework, Ember can't prevent your app from doing algorithmically inefficient stuff. Sometimes you can get away with inefficiencies on the desktop that become deal-breakers on mobile devices, with their constrained CPUs.
The best advice I can give: if you're deploying to mobile devices, get comfortable with the profiler in your browser. It's much better to use a profiler to target the hotspots in your app than to try to prematurely optimize. Usually, that just leads to messy code that is slow anyway.
That being said, many companies are building their business on Ember apps running on mobile devices.
Yehuda: As Tom said, if you try to go low-level in order to get good performance, you may end up with many fast areas of code that combine to produce very slow code. Ember has more bookkeeping overhead, but with a knowledge of the full application, it tends to do a better job of optimizations, like coalescing multiple DOM operations than you would do by hand.
Libraries, like Ember ListView (by core team member Erik Bryn) also provide ways to reuse DOM when working with large amounts of data without giving up the nice APIs of the Ember templating system.
When working in a constrained environment, you will definitely have to focus on performance more. One nice thing about Ember is that, because you're writing to a reasonably high-level API, we can implement improvements that impact all existing Ember applications without needing to ask you to do anything. Over the course of their development cycle, Discourse saw massive performance improvements simply by upgrading to each new version of Ember.
Q With Ember keenly focused on the desktop-like paradigm, what recommendations or resources can you offer developers who want to make the jump into single-page apps?
Tom: The Ember.js guides are great for understanding the framework. We're improving them all the time usually rolling out updates at least once a week and now that the RC is out, we're working hard on material especially designed to get people up and running as fast as possible.
Thanks so much to Yehuda and Tom for taking the time to speak with Nettuts+! If you have any of your questions, leave a comment below, and they just might get back to you!