EasyEngine v4 and updates

Before I get into the current status and future of EasyEngine, I would like to give a brief background of the project and the people behind it.

I believe this will help you better understand the situation by giving you our perspective.

If you want, you can skip to the end.

But first, who is “I”?

I am Rahul Bansal, Founder & CEO of rtCamp. If you didn’t know, rtCamp is the company that develops and maintains EasyEngine. EasyEngine powers all sites managed by rtCamp.

rtCamp is a full-time engagement for me. Like most founders, I am involved in almost every department including sales, marketing, design, engineering, hiring, operations & project management. The unfortunate part is that I have been involved in all these things for last eight years and did not learn to delegate much! ?

Obviously, I enjoy everything I do, and that’s why I do it! The problem is that I don’t get enough time for EasyEngine. As EasyEngine is (still) a side project for rtCamp, I can’t justify spending more time on it as part of my day-to-day activities at work.

I cannot adequately define my role in the EasyEngine project. You could say that I am the Product Lead, but it’s much more than that. I am behind the very idea of the EasyEngine project, its command-line syntax, internal design, architecture, features, and background research.

Of course, I received a lot of help by a team of amazing people who worked very hard to turn my idea into a reality.

However, unfortunately, I am the only one in the EasyEngine team and at rtCamp, who knows both WordPress & Nginx deeply. Most people who joined rtCamp encountered WordPress or Nginx for the first time at the rtCamp!

This makes my job very tedious as I have to spend months on training people. Furthermore, I need to spend considerable time in the hiring process to find these folk.

I often get sandwiched between system admin team (which developed EasyEngine) and WordPress developer team (which needs EasyEngine) as they can’t understand each other’s worlds! These differences started affecting product quality.

I realized a few months back that some radical changes needed to be made. Changes that would delay the v4 release, but will speed up things after V4 gets out.

Before I get into the details, let me quickly summarize the history of EasyEngine as a project to give you a better idea of how we got here!

The EasyEngine story thus far

Let’s have a look at some important events in the life of EasyEngine in chronological order:

  • 2009 – The same year I started rtCamp, I found Nginx when looking for a better web server to handle traffic to our then popular tech blog DevilsWorkshop.org
  • 2010 –  I wrote some PHP scripts which would take a site name as an argument and set up an entire WordPress site- similar to what EasyEngine does. Pragati Sureka improved Those scripts
  • 2010, March 10th – We registered a domain called wpnginx.com and started WordPress-Nginx setup and services. [Side note: The massively successful wpengine.com domain was registered on the same day, just 5 hours later! ? ]
  • 2011 – I spent most of my time working on WordPress-Nginx related problems. I was the only person in our system admin team then. Good part – I got to learn a lot!
  • 2012, Sep  – I published a long series with more than a dozen WordPress-Nginx tutorials. It acted as a base for EasyEngine v1 later
  • 2013, Oct 1st  – EasyEngine v1 released. It was in bash scripting language because Mitesh (our then lead EE developer) was familiar with bash. We released it under rtcamp.com ?
  • 2014, March 15th – I presented EasyEngine at WordCamp Mumbai (blog post). It was very well received. That was the first time I realized that we had created something genuinely useful
  • 2014, July 14thEasyEngine v2 was released. It was a bash to bash refactoring. Mitesh lead it once again with help from Gaurav and Harshad
  • 2014, Nov 13th – EasyEngine.io domain was registered
  • 2015, Feb 11th – EasyEngine v3 was released. Gaurav and Harshad led the release and coded it in Python, as they were comfortable with the language. I was the one who approved their decision to go with Python, so I am the one to blame for this blunder ?
  • 2016, Oct – EasyEngine v4 development started. Another round of refactoring. Finally going back to PHP. I published two posts which covered the reasons for going with PHP, among other things
  • 2017 – We are yet to release v4! I know this is frustrating for many users. Let me give you more details about this next

V4 – Current Status and Release ETA

EasyEngine EE v4 blog post update

EasyEngine was created to make managing WordPress sites on Nginx web servers EASY. If you ask me to describe our #1 goal in a single word, it would be “Easy.” If it’s not Easy, it’s not done!

If the goal had been just to push a release, we could have released EasyEngine v4 a few months ago!

In Dec-Jan, I did an in-depth code review, and I realized that the quality wasn’t up to my expectations. I wanted to review everything that’s going to be in EasyEngine v4 to ensure the best possible quality. I am not happy with the number of issues users face when dealing with Let’sEncypt- one of few features that was released without much of my involvement.

While analyzing the code, I noticed that many common design patterns were not used. That was one of the main reasons why I decided to move the project from the system admin team to WordPress developer team.

Even though both teams are well-experienced, our system admin team has comparatively less programming experience. They are more familiar with writing automation scripts, Ansible playbooks, etc.

It’s very hard to find a competent system admin with good PHP programming skills, at least in India. On the other hand, I am finding it easier to train our best PHP developers about Nginx internals.

What are we working on now?

Even though rtCamp’s WordPress team is busy with client work, they squeezed EE development into their schedules.

It’s going slowly, but things are already looking better. We have worked on many design decisions and developed modular code that can be integrated at a later stage.

Mainly:

  • The “ee stack” command’s codes will be separated as a composer package so other PHP projects can use it. It will make supporting different OS, and even Docker support easy
  • The current mustache templates folder is a mess. We would be refactoring it to use a templating engine for real. This will improve the quality of the Nginx configurations generated. The new templating engine will be available as a standalone site so those who prefer to do things manually, can still have some “easy”-ness (when compared to copying the config from our WordPress-Nginx tutorials section)
  • There are many libraries we are exploring to improve code quality and achieve few things better

Apart from code, we have spent more than 500 man-hours in the last six months researching Docker-based hosting and on two more tasks:

  1. Troubleshooting – Debugging bad-quality code, slow MySQL queries, making sense of error logs
  2. Dev Workflow – Deploying code changes from your development machine to the remote server and vice-versa. Easily creating staging servers from live servers

We always invest a lot more time in background research than code. There are still a lot of problems to get over, but we want to use this refactoring opportunity to build a base to launch them off with minimal efforts in future.

ETA/Release Date

We can’t commit any release date, but we are trying to get v4 out in July 2017.

Of course, you can help us go faster!

Call for Contributors

I am really happy that many in the EE community come forward to help when they saw that development on the project is moving slowly.

At this stage, we require your time more than money!

Below are some areas in which you can contribute. I will move this to a dedicated page soon. If you can figure out any other way to help the project move faster, please let us know via the comments.

  1. Product Lead – Set tasks/features for releases. Dedicate time to test EasyEngine from a user perspective during development. Review code for edge-cases. Ask questions about decisions made. I am ready to spend 10-hours every week, which seems enough for now
  2. Maintainer – Apart from the code itself, we have GitHub issue trackers, the community forum, and the Slack community. Maintainers will be very helpful to manage all of these
  3. Code-Reviewers – Anybody who has experience of reviewing PHP codebase can review pull requests
  4. Architects – If you have spent time designing a Laravel/Symfony project, especially a command-line one, please get in touch right now. We would need at least few hours from you per week for the next 1-2 months. Please have a look at this Github issue for a sample task
  5. Developers – People who can contribute full/part time on programming tasks. To keep the project healthy, we will require one developer full-time. It’s hard for rtCamp to commit a full-time developer for at least the next three months
  6. QA – People who can write Behat-based BDD test cases. We are flexible on Behat as long as you take the lead. CI/CD setup with test cases is also needed
  7. Documentation – Any help with documentation or content would be great. There is so much to document!
  8. Research – Feature requests such as shared hosting, Docker, WordPress-in-subdirectories require a lot of research. But it’s the work done here that makes EasyEngine “Easy”!

As you can see, to manage a project of this size, we need a bigger team. Currently and historically, the core team has been only rtCamp employees. This is not an ideal situation for the project.

One of the main reasons why we decided to refactor EasyEngine to PHP was to leverage the power of the PHP/WordPress community. We need your involvement now more than ever!

What about crowdfunding?

Any financial help would indirectly help as it would enable us to hire dedicated developers for the v4 release. However, I’m not too keen on crowdfunding to release v4 for the following reasons:

  • I don’t think it’s a sustainable model to build something that will require ongoing support. The amount of effort needed after v4 release will be much higher than getting v4 released
  • We might be doing a disservice to the community by running a crowdfunding campaign, as v4 itself won’t have any new features. In fact, it might take away some features!
  • If we decide to run a crowdfunding campaign for the v4 release as well as ongoing development, the target amount might be too large for a community of our size

I am not completely rejecting the idea of crowdsourcing. I feel that there are better alternatives, like recurring donations, something like what patreon.com offers.

If you have any ideas, please let us know. We are not looking for a one-off solution to push us over the line, but a month-on-month recurring flow on an ongoing basis.

What about sponsorships?

Few hosting companies approached us, but things did not work out.

We still would love to have some sponsors as long as their offers are in line with EasyEngine’s project goals.

In Summary

  1. We want EE v4 to be no less than awesome. And that needed us to take a step back [Jump↑]
  2. We require your contribution. For now: Time > Money [Jump↑]
  3. Crowdfunding is not ideal. We are looking for sustainable ways to support the project [Jump↑]
  4. Unless we get some magical contributions, I don’t think we will be able to ship EasyEngine v4 before July 2017 [Jump↑]

All I can ensure is that it will be worth the wait! You can track progress from this GitHub project board.

Do get in touch with us via Twitter or the comments below if you think that you can help the project in any way. We are counting on your support!

Links: EE on GitHub | EE V4 Project board

15 responses to “EasyEngine v4 and updates”

  1. Hope I can help with documentation. Probably start at the part of getting people who are used to shared/managed hosting to as easily as possible get EE going in a VPS.

  2. I am not sure this is relevant to EEv4.0, but we have started a book, documenting sever deployment using GLEEMPLE (mentioned on the EE website here: https://easyengine.io/easyexperts/bamajr/ and here: https://easyengine.io/blog/introducing-bamajr-easyengine-experts/). The book focuses (will focus) on using Google Cloud Platform for VPS hosting and EE for deployment and management. This comment is not intended to be a pitch (as the book isn’t available yet), but perhaps, some documentation could be applicable to your documentation efforts.

    • GLEEMPLE is certainly an interesting idea. ๐Ÿ™‚

      Thanks for using EasyEngine for it. Its this way that makes me go careful about updates as I don’t want to break work done by others.

  3. Hey Rahul, first of all thanks for your great work. This is really important and it’s awesome that there are people like you taking so many hurdles to get this done. My deep thanks for that.

    Unfortunately I’m not experienced in coding in any way which really makes it hard to help. I could help in documentation but don’t know if this would require way more attention that I’m able to contribute at the moment.

    Regarding sustainable financial solutions maybe it would be worth thinking about a licensing model that allowed commercial licences in any way? Maybe as an option? I think most people using EE for commercial reasons would be able to contribute a monthtly licensing fee. This would thinks more sustainable. Sill unsure what this license could exactly look like but maybe a way for the project?

    Thanks again!

    • Thanks for chiming in. Any help will be needed.

      We are exploring options to make documentation process more collaborative.

      I am not sure how well commercial licenses work in command-line world. But I do have some ideas which might be better suited for commercial license. That being said, I would like to keep EasyEngine free and open – as much as possible.

  4. Hey, Rahul!

    I can help you folks out as a code reviewer, but I am not sure about how much time I will be able to contribute towards the project. I have already stepped up my open source contribution time to 50% of all my productive time. Let me know how that will work out, so, that I can plan for it.

  5. Rahul My Friend, I have always been a fan of EE. I just love EasyEngine, and use it in most possible cases. You are an amazingly great programmer with unique combination of skill in both server side and also in PHP. I personally do not believe you did proper balance for EE from rtCamp. This project could have taken far different turn, and could be far bigger. But because of nature of your job and the business and socio-culture we all are in, it did not worked so far.

    But its good that so much work is happening under the wrap. I would love to contribute in any possible way. Documentation and Research or even finding a model to sustain this project in long-run. Would be great if I could find a DevOps from my team to contribute, but I am facing same issue, don’t have any Dev Hour available for next few months.

    Looking forward, Dear. See you soon.

    • @Asif Thanks for your love for EasyEngine โค๏ธ

      I totally agree on not giving enough attention to EE and feel bad about it.

      But you know well about the financial struggle I have been through. I had to choose a focus area and I choose rtCamp. I am glad I did that. Otherwise, rtCamp and EE both would have been history by now.

      Now, rtCamp is growing strong, we hope to deliver some much-awaited updates on EE.

      Thanks for offering to help. I can understand your team being busy. Please ping me whenever you/your team can contribute some time.

  6. | “They are more familiar with writing automation scripts, Ansible playbooks, etc.”

    So why not use Ansible for EE? Looks like Roots used it for Trellis..

    • Ansible is useful for small tasks.

      We are using Ansible internally to perform few system admin tasks on routine basis and on-demand across servers we manage.

      The simplicity of EasyEngine and its current, as well as future feature-set, require more robust software development environment approach which is beyond the scope of Ansible.

      That being said, EasyEngine is open to use Ansible for few tasks, but the main project will be in PHP, using Laravel or Symfony framework.

  7. Hey Rahul

    I could translate the documentation to german. I think this would allow many new users to use easy engine!
    Or: I would help create the “normal” documentation.