Picking the right code review tool for your organization

by Phil Miller

‍

It’s well established that code reviews are an important part of any healthy software engineering organization. We’ve even recently covered tips for how to improve your reviews and covered all the popular tools that power them. 

Given all of that, let’s look at how to select the best tool for your organization, both today and in the future. 

‍The goal here is not to tell you what tool to pick, but to give you a framework for asking the right questions so that you can decide for yourself. 

Putting your needs first

The first thing to consider is the current scale of your engineering organization. Your needs will be quite different depending on whether you’re early-stage, mid-size, or enterprise scale. You also need to consider the size of your codebase(s), the number of contributors, and whether you have in-house resources that will be responsible for any related infrastructure. 

Next, think about the current state of code review in your organization. What are your current challenges? Are you even doing formal code review today, or is it more of a rubber stamp? If you’re not doing reviews, what risks does that pose and how will it evolve over time? 

Finally, think about your technical roadmap. Will code review tools help enable anything on it? For example, if you’ve promised a large customer that you will be SOC 2 compliant by year end, then this may influence the type of tools and processes for code review. 

Key Features

Once you have your organizational needs in mind, consider the required features in terms of: 

Integrations: 

• What third party tools are crucial to your development process?

Customization: 

• How important is workflow customization and extensibility? 

User Experience

• What kinds of devices and interactions do you expect your team to need? 

Collaboration and Communication

• How does all of this fit with your corporate culture for team interaction? 
• What types of collaborative features are the most important?

Security and Compliance

• What are your needs in terms of access control, permissions, authentication, etc?
• Is this tool compatible with any certifications that are a legal or regulatory requirement for your organization? 

Evaluating solutions

Once you have some initial answers to the above questions, go back and look again at our previous article about code review tools. Think about which one might be the best fit given all of your considerations. 

Now it’s time to plan how to effectively evaluate the tool(s) you have chosen. It’s worth noting that it’s very hard (or potentially impossible) to trial multiple solutions for code review on the same repository. As such, you may want to consider multiple consecutive trials, or parallel trials on different repos. Just make sure that you are evaluating them in a similar fashion. 

To do this effectively, consider the following: 

Demos and Trials

• Is the length of the trial long enough to truly evaluate the solution? (e.g. Can you get in at least one sprint/cycle?) 
• Is the trial permissive enough in terms of feature access? (e.g. Are you able to access all of the features you require or are they gated behind a payment?) 

Reviews and Case Studies

• Are there reviews or testimonials for organizations similar to yours? If not, can you ask for a reference from an existing customer that has similar needs?
• What case studies or whitepapers are available?

Pricing and licensing 

• Do you need a perpetual license or is this subscription based? How does this impact support plans?
• How will your integration, automation, and infrastructure needs affect total cost? 
• How much more will this cost when the license/subscription renews, given your projected headcount growth? 

Making the call and rolling it out

So you’ve gone through demos, a trial (or two), and you’re ready to commit. Now is the time to make sure you’ve got an effective plan to roll this out to your organization. Hopefully, you tackled setting up all the integrations and workflows during the trial period, so that shouldn’t be a concern at this point. You can focus on training your teams, and teaching them how to do an effective code review in your shiny new tool. 

Because you (hopefully) just finished an evaluation during the trial phase, you should have a sense of your KPIs and mechanisms for feedback. This is the perfect time to make those a regular thing in your overall process and reporting. When you continuously evaluate something like code reviews, you will always have an indicator of what is and isn’t working. So if you need to re-evaluate your set of tools in the future, you will already have a very strong foundation to do so. 

Conclusion

We hope this provides you with a solid framework for evaluating your code review tooling today and in the future. If you are currently suffering the business risk of not doing reviews, the best time to start is today. 

If all of this feels a bit overwhelming, don’t worry you’re not alone. Trying to force an organizational change for something as fundamental as code reviews can be a monumental task. If you want to try something that can level up your reviews today, we built the Korbit AI Mentor for this exact reason. Please give it a try and tell us what you think! 

‍