How to format FIM prompts with multiple imported files for context?

[klakovsky]

I'm trying to use Seed-Coder for Fill-in-the-Middle inference and want to understand the proper way to format prompts when including multiple files from imports along with the current file.

According to the paper, during training you used different approaches for multi-file context:

• Repository-level data with topological concatenation based on file dependencies
• Commits data with BM25-retrieved top-5 relevant files
• CrossCode evaluation with cross-file context

However, I couldn't find clear documentation on how to format the inference prompt when I have:

1. A current file with imports
2. Multiple imported files that I want to include as context
3. A specific location where I want the model to fill in code

Example

# utils.py
def helper_function():
    return "helper"

class DataProcessor:
    def process(self, data):
        return data.upper()

# config.py  
DATABASE_URL = "sqlite:///app.db"
DEBUG = True

# main.py (current file)
from utils import helper_function, DataProcessor
from config import DATABASE_URL

def main():
    # <-- WANT TO FILL HERE
    return result

Request

Could you provide:

• Official guidance on multi-file FIM prompt formatting
• Examples of best practices
• Any specific format that was used during training
• Token limit recommendations when including multiple files

Thank you!

[klakovsky]

Currently trying to format like this:

# utils.py
def helper_function():
    return "helper"

class DataProcessor:
    def process(self, data):
        return data.upper()

# config.py
DATABASE_URL = "sqlite:///app.db"
DEBUG = True
<[fim-suffix]>
    return result

<[fim-prefix]># main.py
from utils import helper_function, DataProcessor
from config import DATABASE_URL

def main():
    <[fim-middle]>

It sometimes outputs filepath instead of correct completions.

[Nes-tle]

FIM should always be prompted like a single doc. If more than one file is required, then you need to concatenate them before formatting as FIM. Like:

<[fim-suffix]>File2suffix, File3, File4<[fim-prefix]>File1,File2prefix<[fim-middle]>

[klakovsky]

Thanks for the clarification! I have a follow-up question about file concatenation format.
When concatenating multiple files, should I include filenames/paths as separators? If yes, what's the recommended format?

Should I use:

• Just filename: # config.py or // Config.java
• Relative path: # src/config.py or // src/main/java/Config.java
• Or no file separators at all?

[kotlyar-shapirov]

FIM should always be prompted like a single doc. If more than one file is required, then you need to concatenate them before formatting as FIM. Like:

<[fim-suffix]>File2suffix, File3, File4<[fim-prefix]>File1,File2prefix<[fim-middle]>

In the example you mentioned that File3 and File4 goest after suffix token and File1 goes after prefix token. How do you decide if context File goes to suffix or prefix?

[Nes-tle]

For the recommended format, the one with relative path is preferred, which is more commonly used in the prompt for general coders.

For the file order, File1, File2 is to indicate some predefined order in the example. In some languages such as C++, the reference graph may not necessarily be a DAG, so the order remains flexible. If you are going to test it on some repo-level benchmarks, which typically involve completion or generation in a single file, my suggestion would be that all relevant files should be assigned to the prefix part to provide enough context. Meanwhile, if you want to explore customized long context training or plugin development, then the best way of repo organization remains an open problem.

Hope this note can help!

[g0t4]

TLDR: try Qwen2.5-Coder's repo-level FIM

I have repo-level FIM working with Qwen2.5-Coder's repo-level format and tokens! Make sure to set stop tokens or it will ramble, notably <|file_sep|>, <|end|> and <|end_of_text|>... I've been using this with RAG for auto context from my repos and Seed-Coder successfully incorporates RAG matches into the FIM completion!

i.e. prediction on left, stopped appropriately and it's M.div lua function matches that of another similar div function (right side) in a different file in my repo (the RAG match)

YMMV, I will do more testing and see if this reliably works.

Repo level mentions in Seed-Coder paper

Ok so... this is in their technical paper:

Each repository was mapped to a single string sequence

with exceptionally large repositories (e.g., PyTorch) being decomposed into multiple independent subgraphs

repository-level variant preserved the project structure

RepoCoder reference

Seed-Coder paper links to evals from a RepoCoder paper... RepoCoder is a framework for REPO LEVEL code generation using iterative RAG+gen. https://arxiv.org/pdf/2303.12570

Here's an example prompt... note it is commented out and I tried this format with Seed-Coder and while it worked it didn't reliably stop and I doubt anyone would train a model with commented out code for repo-level context.

https://github.com/microsoft/CodeT/blob/main/RepoCoder

"Repo-Level Codes"

Below it says Repo-Level Codes... right after File-Level Codes... that smacks of the same training used for Qwen2.5-Coder (and many others)... so can't we just use that with a FIM at the end?