[Bug]: Editing does not work on lines that contain \n

[neubig]

Is there an existing issue for the same bug?

• I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• I have checked the existing issues.

Describe the bug

I have found an easy-to-reproduce error in our editing pipeline that should be a big win if we can fix it: editing seems to break on lines that contain \n.

Here's an example:

• Prompt: Edit the file multiline_print.py to add a markdown link to https://github.com/All-Hands-AI/OpenHands around the word "OpenHands".
• File:
  multiline_print.py.txt
• Result: https://www.all-hands.dev/share?share_id=0d316598dc8aa26bff4ab49c69b44d705c3a778765d0656394e0bb4c8c836545

Current OpenHands version

c875a5fb777fcdb387df13d78789a5597f3d29e3

Installation and Configuration

make build; make run

Model and Agent

• Model: claude-3-5-sonnet
• Agent: CodeActAgent

Operating System

Mac OS

Reproduction Steps

No response

Logs, Errors, Screenshots, and Additional Context

Part of #3231

[shubhamofbce]

I tried it using gpt-4o and it worked for me. Intendation was broken after the edit but it fixed that by itself.

Output:

[neubig]

Interesting! So it may be a claude problem to some extent. But actually @shubhamofbce it didn't really work with gpt-4o either, because gpt-4o deleted the newlines entirely (which was not requested).

[shubhamofbce]

I see the problem now, It removed the newlines(\n) completely. I will take a look at this once i get some time.
I am currently working on #3575

[neubig]

Awesome, thanks!

[xingyaoww]

I think this may also be fixed if @RajWorking can implement a EditAction using the Aider Diff format

[shubhamofbce]

Then we can assign this to @RajWorking and I will look into something else.

[tobitege]

This is being worked on by @RajWorking

[RajWorking]

I am working on this. Thanks!

[tobitege]

@neubig - this issue is important to get fixed asap, imo.
cc @RajWorking @xingyaoww @enyst

Earlier today I was running ~15 instances of the Aider bench (which previously also failed) and was wondering why the edit_file_by_replace was often not successful in finding the text-to-be-replaced without apparent reason (no text cut off).

As raised in this very issue, the \n is converted incorrectly:

LLM (Sonnet) action:

15:54:00 - openhands:DEBUG: stream.py:134 - Adding IPythonRunCellAction id=7 from AGENT
15:54:00 - openhands:INFO: agent_controller.py:489
ACTION
**IPythonRunCellAction**
THOUGHT: Thank you for the update. Let's continue refactoring the code. We'll now update the `format_entries` function to use the new `Currency` class and improve its structure:
CODE:
edit_file_by_replace(
    'ledger.py',
    to_replace="""def format_entries(currency, locale, entries):
    if locale == 'en_US':
        # Generate Header Row
        table = 'Date'
        for _ in range(7):
            table += ' '
        table += '| Description'
        for _ in range(15):
            table += ' '
        table += '| Change'
        for _ in range(7):
            table += ' '

        while len(entries) > 0:
            table += '\n'

            # Find next entry in order
            min_entry_index = -1

---

OBSERVATION with error (sorry, but the formatting gets a bit disrupted here):

15:54:00 - openhands:DEBUG: stream.py:134 - Adding IPythonRunCellObservation id=8 from AGENT
15:54:00 - openhands:INFO: agent_controller.py:215
OBSERVATION
**IPythonRunCellObservation**
[No exact match found in ledger.py for
"""
def format_entries(currency, locale, entries):
    if locale == 'en_US':
        # Generate Header Row
        table = 'Date'
        for _ in range(7):
            table += ' '
        table += '| Description'
        for _ in range(15):
            table += ' '
        table += '| Change'
        for _ in range(7):
            table += ' '

        while len(entries) > 0:
            table += '
'

            # Find next entry in order
            min_entry_index = -1
"""

This looks like a common contributor for failed instances just because the step-limits fail and/or loop detection gets triggered?

[enyst]

Ah, I see the issue with '\n'. But the indentation is actually correct?

[tobitege]

It was a formatting issue in the comment, changed some triple-backticks, looks little better now 😄