The New Toolkit for AI Software Developers

Creator: Andrew Barkett
Editor: Phil Miller

‍

In today's fast-evolving technological landscape, artificial intelligence (AI) has significantly transformed how developers approach their tasks and projects. It's imperative for both managers and developers to stay updated with the "new toolkit" that integrates AI to enhance efficiency and productivity. Korbit AI is part of a broader suite that powers today's cutting-edge development processes. Below, we'll explore a typical set of tools used by a startup creating a new AI-driven product.

Macbook Pro:

Description: For many developers, a Mac is a good balance between a convenient user experience and a Linux/UNIX-like environment that’s closer to the server deployment target.

Relevance: The neural cores in M1/M2 chips are not bad for AI-specific tasks, and 24-32G of RAM is now essential.

Costs/Benefits: Let’s say the computer is 70 per year for Applecare.  All in, we’ll call this $56 per month.  The benefits here are straightforward: you need a local environment for coding and debugging, and ideally one in which you can brew or pip install lots of things, run docker, or run emulators or other VMs as needed.

AWS/Azure/GCP:

Description: These are leading cloud platforms offering a myriad of services for computing, storage, and AI capabilities.  If you’re not training your own LLM, you probably don’t need GPU-heavy machines.  If you do, costs go up.

Relevance: The scalability and flexibility of cloud infrastructures are paramount in using and deploying AI models, making these platforms indispensable.

Costs/Benefits: Varies greatly based on specific services used and where you are in the process, but let's estimate an average of $100 per month for moderate usage. This doesn’t include your full production-serving infrastructure.  Benefits include scalability, reduced operational costs, and streamlined development processes.

vscode:

Description: A lightweight, extensible source-code editor.

Relevance: Enhances coding efficiency with AI-powered extensions and integrations.

Costs/Benefits: Free, but let's say $10 per month for premium extensions (not counting Copilot). Benefits include faster code writing and debugging.

Github/Bitbucket/GitLab/Azure DevOps:

Description: Platforms for version control and code collaboration. See our code review tools post for more on these

Relevance: Facilitate collaborative development and transparent versioning of models and data processing pipelines.  Korbit, and other tools, also integrate directly with Github, Bitbucket, and other systems, so your “AI resources” use these platforms as a way of communicating with your human resources.

Costs/Benefits: Typically 15 per user, depending on usage; core benefit is streamlined versioning and collaboration, even between human and robot.

Korbit AI Mentor:

Description: A virtual senior engineering resource to automate and augment various software engineering tasks.

Relevance: Reduces development time and costs by leveraging AI for coding assistance, code review, debugging, policy-enforcement, and problem-solving.

Costs/Benefits: $40 per month; benefits include technical debt reduction, less hours spent in code reviews, better code quality, less bugs reaching production, better support for junior engineers, manager visibility into the team and code quality, and faster cycle times.

Datadog:

Description: Monitoring and analytics platform for cloud-deployed applications.

Relevance: Monitors AI and traditional applications in real-time, ensuring optimal performance and uptime.

Jupyter:

Description: An open-source platform for creating and sharing live code, statistical analysis, and visualizations.

Relevance: Essential for interactive AI model development and data analysis.  Also useful in LLM-usage and quality evaluation.

Costs/Benefits: Free/included, assuming you’re running on your local machine/cloud resources; benefits encompass interactive model development and ease of data exploration.

Gmail/Google Docs/Google Meet:

Description: A suite of productivity tools for communication and collaboration.

Relevance: Facilitates seamless team coordination during traditional and AI-driven projects.

Costs/Benefits: Roughly $15 per month; benefits are real-time collaboration and enhanced team communication, plus your customers have to have an email address to email when they want to order your products!

Copilot:

Description: An AI-powered coding assistant from Github.  Integrates well with VS Code.

Relevance: Provides coding suggestions and auto-completion as you type, making the development process faster and more efficient.

Costs/Benefits: $19 per month; the chief benefit is accelerated code writing. It now includes Copilot Chat. 

ChatGPT Plus / OpenAI APIs:

Description: LLM Foundation Models and Advanced AI tools for multiple tasks.

Relevance: Aids in automating customer interactions, knowledge base queries, debugging, generating documentation, and more.  The possibilities are endless.  The gpt-4 is still pricey, but its capabilities far outstrip the gpt-3.5-turbo.

Costs/Benefits: ~$100 in LLM usage; benefits include increased employee speed, enabling classes of applications that were not previously possible, and greatly reducing time spent translating between human language and machine instructions.

Slack:

Description: A messaging platform for teams.

Relevance: Streamlines communication in AI development environments where instant feedback is essential.  Includes access to many bots and easy integrations.

Costs/Benefits: ~$10 per month; primary team communication tool.  Better than email alone.

‍Jira/Linear:

Description: A project management and issue tracking tool.

Relevance: Organizes AI development cycles, ensuring timely delivery and issue resolution.

Costs/Benefits: ~$10 per month; core benefit is structured project management.

Pagerduty:

Description: An incident response platform.

Relevance: Ensures AI services remain up and running by alerting teams to issues in real-time.

Costs/Benefits: Expect $20 per month; benefits are quick incident response and uptime assurance.

In Summary:

The era of AI-driven development has brought a plethora of new tools to the table and made some existing tools more or less relevant. While some of these tools come with a price tag, their benefits often outweigh the costs. Based on the above estimates, including amortizing their laptop, it would cost about $150/month when you’re just starting out on a new venture, and double it if you’re in a larger company that has lots of other tools you need to use.

This investment, however, paves the way for faster development cycles, reduced operational costs, enhanced team communication, and more optimized AI applications. For organizations aiming to stay at the forefront of technological advancement, this toolkit represents an essential and worthwhile investment.  An AI-forward toolkit is also a selling point and retention mechanism for engineers who want to work on cutting-edge topics with cutting-edge tools.

In my experience as an engineering leader at Google, Facebook/Meta, NVIDIA, and many startups, engineers with the right toolkit can be anywhere from 20-80% more efficient and effective.  At Google back in the 2000s, much of this toolkit was home grown.  Today, off-the-shelf tools like AWS, Copilot, Jira, and Korbit’s AI Mentor give you a toolkit we couldn’t have imagined just a few years ago, and make engineers ultra-productive, at an all-in cost of less than $500 per month.

What parts of the modern AI developer toolkit are you missing? If one of them is an AI Mentor for your engineers, you should definitely try Korbit!

‍