This is Part 2 of the Working with AI series. Read Part 1: Understanding the Art of the Prompt.

When it comes to software development at Mantle, we always strive to challenge the status quo. We were faced with a large but common challenge: turning a prototype project into a production project in the style and structure of our other existing production code.

Once we saw Gemini’s release of 1.0 Pro with a one million token window, we developed an approach that reduced the scope by two-thirds and saved months of developer time by using an LLM to assist this large engineering task.

The Challenge

There is a recurring need in the software world for teams to convert a codebase from one language to another. These initiatives always seem like a daunting task, but there are several compelling reasons why organizations take the plunge:

Improved Maintainability: Legacy codebases can become cumbersome and difficult to maintain, especially if the original maintainers are no longer contributing. Switching to a language with a larger knowledge pool can make the code easier for developers to understand and modify.
Performance Boost: Different languages excel in different areas. If a team finds their application is struggling to meet performance benchmarks, migrating to a language known for speed or efficiency can provide a significant boost.
Talent Pool: Finding developers with expertise in a particular language can be challenging. If the current language has a limited talent pool, migrating to a more common one can make it easier to recruit and retain skilled developers.
Productizing Prototypes: Concepts that might be written in languages/frameworks that are quick and easy to prototype a solution, but aren’t well suited for production use cases.

When these types of pressure points arise, there’s a common understanding as to why the effort to convert is justified, yet a shared acknowledgement that the process will be painful. Instead of moving a customer-facing roadmap forward, you are now going to be spending a significant portion of valuable engineering time recreating existing functionality. Also, it’s no secret that these project timelines often go beyond estimates and are riddled with unforeseen issues and risks.

At Mantle, we came up with a different approach.

Our team had a prototype written in the language, R, and wanted to convert this to our standard production tech stack, Golang and ReactJS. The goal was to take the prototype’s logic and intent and use an LLM to translate the R code using our production code patterns, file structures, and libraries.

Our goal wasn’t to achieve 100% perfectly crafted code. The goal was to get 80% of the boilerplate and repeated patterns out of the way so that engineers could focus on the 20% – high value polish to ship the project to customers.

Gathering context

While generating working code from an idea or instruction has been possible for a while now, generating large scale codebases written in our style and structure, and connected to common utilities, is something that’s only recently been enabled thanks to the latest generations of Gemini LLMs and their >1 million token context windows.

There are ways to cope with smaller context windows when reasoning about codebases, but larger context windows allow you to put full codebases into a prompt with additional instructions/context with ease. We took full advantage of this in many ways.

Inputting prototype source code

We inserted the existing prototype codebase as context into our prompt by giving a bit of text explaining what it is and then injecting the source in a human-readable way.

Context: Here is a list of files from an application written in typescript.

file: ./app/R/components.R

<source code for components.R>

file: ./app/R/table.R

<source code for table.R>

This format of source code and file path is an important nuance as we ask to output individual files later on in the process. Where they sit in relation to each other is important context as we generate files.

Derive existing code patterns

With this context, we could have started generating code, but the style and approach could vary wildly by file. The output code could also be written in a way that the LLM seems adequate based on its training data.

In our initial attempts, where we included existing production source code from the target language as a reference, showed that it would too heavily influence the output. The solution we used was to abstract the code patterns out of the production source code using a prompt.

Creating a summary prompt can generate code that would use the patterns in the code below that includes its code structure and patterns.

<target language source code>

The output from this prompt would then be added to our context using the file path format that can be used to associate the files we generate later on in the process.

Context: Here is an overview of the target architecture:

file: ./src/utils/time.ts

<summary prompt output>

This works well with individual files that you want to give context on or it also works well with giving context to wildcard file paths (e.g. file: ./src/utils/*.ts) which is very convenient.

Existing libraries

We have certain libraries that we prefer, so adding a section of context with the current libraries from existing codebases was very helpful to make sure the LLM outputs code that is compatible with them. In this case, we insert the go.mod and package.json file from one of our existing codebases directly into the context.

Context: Here is a list of libraries that are available to use.

< go.mod >
< package.json >

Screenshots as visual reference points

When dealing with frontend code, we use the screenshots to help reinforce what is needed from the output. This is especially helpful when dealing with frontend code where the UI is generated by third party libraries, so layout specifics may not be clearly defined in the code. Screenshots of the existing application give the LLM a visual layout of the application.

Reinforce with already generated or ported code

As we generated code file-by-file, we included it in the context so the LLM always has a view of the current state of the codebase we’re generating. This helps a LLM make decisions to connect and build off of the code that is already there.

While iterating on this process, we found that adding any portable internal libraries or utilities from other existing projects as early as possible was very helpful. Otherwise, you might find that it recreates abstracted and refactored code that already exists.

Here is some code that's already been generated:

file: ./src/utils/time.ts
< source code >

Compiling context is a complex but crucial task in order to recreate a holistic view for reasoning about code. This process mirrors how developers approach tasks, but the information needs to be formatted in a way that is compatible for an LLM.

In terms of overall input token usage, the prototype codebase we worked with was ~43,379 tokens (12,137 lines code). Taking into account everything else, this amounted to about ~80,000. As we started generating files and approached feature parity with original code, our usage was >500,000 tokens. There’s definite room for optimization here, and the amount of context needed will vary depending on the programming languages and frameworks involved.

Generating Files

At this point, we had taken full advantage of the large token input limits to set up all the necessary context to reason about the problem. However, due to the limitation on output (often around 8,192 tokens), we had to generate files one by one, or sometimes, section by section.

With all the context from the previous section, the general cycle of code generation looked like this.

Add any additional context that might be helpful for the specific file you want to generate.
Generate some code.
Decide whether to re-generate with additional context or tidy up any imperfections in the source code and add it to the context.
Rinse and repeat.

The Approach

We found that the best way to approach the generation is from backend to frontend and starting at leaf node (bottom-up tree traversal) files (utilities, libraries, database layer, etc) then up to the more common and connected (API interface or routing) ones. This gives the LLM a way of triangulating which files relate to each other and decide when it’s appropriate to import and use them.

Executing the Prompt

In our case, the prototype app had a database schema we wanted to build up from. This helped us indicate if we were getting closer to feature parity since both apps could be run side-by-side and pointing at the same data source.

When beginning this process from scratch, the first files had little context to work off of. It helped to add some extra context to help educate the LLM towards a good outcome. We did this by adding the database schema to the context, then generating the database models for our application.

Here is the database schema for the generated code:

<database schema>

Now we add the code conversion instruction that will be used when generating files.

Prompt: You are a software engineer and you can only answer with source code. Convert the provided <prototype code language> application code to a <Production language> application, maintaining a similar structure and functionality as the given example and using libraries provided. Output the full source code without explanations.

Output ./server/db/models/company.go

Once the file was generated, our team either reviewed and adjusted the output manually or adjusted the prompt to recreate the output. It’s important to catch any patterns early on before they compound and multiply, so taking this step to review is critical. Once we were satisfied with the output, we added it into the overall context so that the newly generated file could reinforce the next file we created.

Dealing with Large Files

In our prototype app, the database querying layer was quite verbose and greatly exceeded an LLM’s output token limit. We slightly adjusted the approach in order to generate complete segments of files that we could concatenate together in our context.

To do this, we adjusted the output line of our prompt.

Output the interface for ./server/db/manager.go

An example of the output looked something like this.

// Companies

CreateCompany(ctx context.Context, org dbmodels.Company) (dbmodels.Company, error)

GetCompany(ctx context.Context, id uuid.UUID) (*dbmodels.Company, error)

UpdateCompany(ctx context.Context, org dbmodels.Company) error

DeleteCompany(ctx context.Context, orgId string) error

ListCompany(ctx context.Context) ([]dbmodels.Company, error)

After reviewing and adding this partially created file to the context, we did follow-up prompts to help generate the rest of the file and while keeping within the output token limit.

Output the implementation of Companies for ./server/db/manager.go

Repeating this process, we were able to generate the implementation of the entire interface and then move on to the next file. With token windows evolving quickly, hopefully this step won’t be necessary in the near future.

Conclusion

This article explored how Mantle leveraged LLMs streamline code conversion from prototype to production. By utilizing the model’s capabilities to generate code with context, we were able to save two-thirds of the time needed for our task of converting a codebase to another language.

Using these techniques, the approach taken will have to differ depending on the context you have around you. This article is a glimpse into how you can use things like folder structures, code patterns, and/or adjacent code/schemas as context to complete a number of coding tasks using LLMs.

As token windows continue to grow and models become more adept at understanding and generating code, there will be even greater efficiencies and improved code quality in the conversion process. This will undoubtedly lead to more rapid and cost-effective software development, which will benefit the whole ecosystem.