Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TextInput cursor lags behind text when component is controlled if Auto-Correction and Predictive Text are enabled #44157

Closed
mozzius opened this issue Apr 19, 2024 · 14 comments
Labels
Component: TextInput Related to the TextInput component. Issue: Author Provided Repro This issue can be reproduced in Snack or an attached project. Newer Patch Available Resolution: Fixed A PR that fixes this issue has been merged.

Comments

@mozzius
Copy link

mozzius commented Apr 19, 2024

Description

Whilst typing very fast, we've observed that the cursor can lag behind where you are typing, which means that characters get inserted in the middle of the word. We believe that we've narrowed it down to something with the autocorrect/predictive text messing up the cursor position. This only happens to fully controlled inputs - specifically if passing it the value prop. If value is not passed, it seems to be unaffected.

Going into Settings > General > Keyboard and turning off both "Auto-Correction" and "Show Predictions Inline" resolves the issue entirely, leading us to believe it's something to do with the iOS 17 inline predictive text. However, we've found that disabling autocorrect via the autoCorrect prop does not consistently fix the issue, although it improves it ever so slightly.

We've observed this in production on iOS, but is a lot easier to replicate on the iOS simulator (presumably since it's slower).

Steps to reproduce

  1. Install the app on the iOS simulator
  2. Type as fast as you possibly can into both text inputs
  3. Observe the top text input's cursor is not at the end of the text, and the bottom text input's cursor is at the end of the text
  4. Go to Settings > General > Keyboard and turn off both "Auto-Correction" and "Show Predictions Inline" and repeat - both inputs should now behave properly with the cursor at the end of the text

Note that the spam typing does not have a 100% success rate, so you might have to try a couple of times

React Native Version

0.75.4

Affected Platforms

Runtime - iOS

Output of npx react-native info

System:
  OS: macOS 15.0
  CPU: (8) arm64 Apple M1
  Memory: 883.95 MB / 16.00 GB
  Shell:
    version: 3.7.1
    path: /opt/homebrew/bin/fish
Binaries:
  Node:
    version: 22.9.0
    path: /opt/homebrew/bin/node
  Yarn:
    version: 3.6.4
    path: /opt/homebrew/bin/yarn
  npm:
    version: 10.2.5
    path: /opt/homebrew/bin/npm
  Watchman:
    version: 2024.09.30.00
    path: /opt/homebrew/bin/watchman
Managers:
  CocoaPods:
    version: 1.15.2
    path: /opt/homebrew/bin/pod
SDKs:
  iOS SDK:
    Platforms:
      - DriverKit 24.0
      - iOS 18.0
      - macOS 15.0
      - tvOS 18.0
      - visionOS 2.0
      - watchOS 11.0
  Android SDK:
    API Levels:
      - "33"
      - "34"
    Build Tools:
      - 30.0.3
      - 33.0.0
      - 33.0.1
      - 34.0.0
    System Images:
      - android-34 | Google APIs ARM 64 v8a
      - android-35 | Google Play ARM 64 v8a
    Android NDK: Not Found
IDEs:
  Android Studio: 2023.1 AI-231.9392.1.2311.11330709
  Xcode:
    version: 16.0/16A242d
    path: /usr/bin/xcodebuild
Languages:
  Java:
    version: 11.0.24
    path: /opt/homebrew/opt/openjdk@11/bin/javac
  Ruby:
    version: 2.6.10
    path: /usr/bin/ruby
npmPackages:
  "@react-native-community/cli": Not Found
  react:
    installed: 18.3.1
    wanted: 18.3.1
  react-native:
    installed: 0.75.4
    wanted: 0.75.4
  react-native-macos: Not Found
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: true
  newArchEnabled: false
iOS:
  hermesEnabled: true
  newArchEnabled: false

Stacktrace or Logs

n/a

Reproducer

https://github.com/mozzius/laggy-textinput-repro

Screenshots and Videos

Simulator.Screen.Recording.-.iPhone.15.-.2024-04-19.at.01.55.32.mp4
Copy link

⚠️ Newer Version of React Native is Available!
ℹ️ You are on a supported minor version, but it looks like there's a newer patch available - 0.73.7. Please upgrade to the highest patch for your minor or latest and verify if the issue persists (alternatively, create a new project and repro the issue in it). If it does not repro, please let us know so we can close out this issue. This helps us ensure we are looking at issues that still exist in the most recent releases.

@github-actions github-actions bot added Newer Patch Available Component: TextInput Related to the TextInput component. labels Apr 19, 2024
@cortinico cortinico added Issue: Author Provided Repro This issue can be reproduced in Snack or an attached project. and removed Needs: Triage 🔍 labels Apr 19, 2024
@cipolleschi
Copy link
Contributor

Hi @mozzius, thanks for the issue and also for the reproducer, that's super valuable.

FWIW, I have seen the cursor jumping also in fully Native apps. We will surely investigate the issue, but it could also be something on Apple side, especially if you mention that this is starting happening in iOS 17 specifically.

Would you be able to test it on an iOS 16 device?

@mozzius
Copy link
Author

mozzius commented Apr 19, 2024

Here's an 16.4 simulator - surprisingly, it's present here too. It's definitely still the autocorrect though - turning off "Auto-Correction" and "Predictive" fixes it.

Screen.Recording.2024-04-19.at.15.21.48.mov

@megantaylor
Copy link

So happy to come across this issue, as I've been assuming I was somehow doing something wrong in my uses of TextInput while trying to debug this problem in my app.

While researching, I was reminded of this change from a couple of years ago: https://reactnative.dev/blog/2021/09/01/preparing-your-app-for-iOS-15-and-android-12#quicktype-bar

Could totally be a red herring, but just thought I'd throw it out there in case its somehow relevant. I tried testing inputs with spellcheck disabled, but due to the inconsistency of the bug, I'm having a hard time deciding if it's actually making a difference or not.

@hezi
Copy link

hezi commented May 2, 2024

Took some time to look at this yesterday, seems to boil down to -[RCTTextInputComponentView _setAttributedString:] not keeping up (specifically position calculation) with user when spamming keys while autocorrect/spellcheck is involved (quick way to test: spam just numbers and it works perfectly.)

I'll set aside some time today or tomorrow to dig deeper, but if anyone wants to give it a go - you are more than encouraged to do so :)

I'm curious though: @mozzius @megantaylor - Are you seeing this happen with "normal" user typing? I couldn't repro when typing normal text on a phone.

@megantaylor
Copy link

@hezi yes, we have been able to repro with "normal" typing on actual devices.

@haileyok
Copy link
Contributor

Hey @hezi!

@mozzius may have missed this, was cleaning up some tickets and was reminded of the issue. We definitely have seen this on device as well, albeit not nearly as noticeable. Sometimes though the cursor will indeed skip behind a character or two, though you do need to be typing fairly quickly to notice it.

Appreciate you taking a look at this!

@haileyok
Copy link
Contributor

Probably worth noting that this was more prevalent on some older iPhones from my experience. It is much easier to produce on an iPhone X for example than my 14, though it does produce on both.

@enmanuelmag
Copy link

I have a iPhone 14 pro and the bug raise too. Also when I type on iOS simulator. Any work around or solution?

@GuoXiaoyang
Copy link

Is there any progress?

@gaearon
Copy link
Collaborator

gaearon commented Oct 5, 2024

I'd like to propose deprecating controlled inputs in RN (at least in the docs) until this can get prioritized and fixed.

facebook/react-native-website#4247

@mozzius
Copy link
Author

mozzius commented Oct 5, 2024

updated repro to 0.75.4, confirmed it's still happening

@lorenc-tomasz
Copy link

@cipolleschi Today I have found this issue: facebook/react-native-website#4247 and when you type very fast it happens. Was not aware of this issue, but now its super annoying :D Any idea how to fix this?

@cipolleschi
Copy link
Contributor

@lorenc-tomasz we are currently looking into it and we have a couple ideas on how to fix it. The fix will probably arrive in 0.77. The problem is an hard one, although it doesn't look like, and the current solution arrived after several trade-offs we evaluated in the past to balance perfomance, ergonomics and other aspects.
That said, we are committed to fix it and we are actively working on it: the root cause seems to be Apple's autocorrection that does not behave well when it tries to replace the text with the correction.

The issue is also not as grim as it looks: you need to type really fast to trigger it and this happens on the simulator or on a real device connected to a keyboard. Real devices using the software keyboard are unlikely to hit the problem as it is quite hard to type that fast.
If you are still encountering the problem, the suggestion is to avoid using a controlled input.

NickGerleman added a commit to NickGerleman/react-native that referenced this issue Oct 14, 2024
…n controlled single line TextInput on iOS (New Arch) (facebook#46970)

Summary:

Fixes facebook#44157

This one is a bit of a doozy...

During auto-correct in UITextField (used for single line TextInput) iOS will mutate the buffer in two parts, non-atomically. After the first part, after iOS triggers `textFieldDidChange`, selection is in the wrong position. If we set full new AttributedText at this point, we propagate the incorrect cursor position, and it is never restored.

In the common case, where we are not mutating text in the controlled component, we shouldn't need to be setting AttributedString in the first place, and we do have an equality comparison there currently. But it is defeated because attributes are not identical. There are a few sources of that:
1. NSParagraphStyle is present in backing input, but not the AttributedString we are setting. 
2. Backing text has an NSShadow with no color (does not render) not in the AttributedText
3. Event emitter attributes change on each update, and new text does not inherit the attributes.

The first two are part of the backing input `typingAttributes`, even if we set a dictionary without them. To solve for them, we make attribute comparison insensitive to the attribute values in a default initialized control. There is code around here fully falling back to attribute insensitive comparison, which we would ideally fix to instead role into this "effective" attribute comparison.

The event emitter attributes being misaligned is a real problem. We fix in a couple ways.
1. We treat the attribute values as equal if the backing event emitter is the same
2. We set paragraph level event emitter as a default attribute so the first typed character receives it

After these fixes, scenario in facebook/react-native-website#4247 no longer repros in new arch. Typing in debug build also subjectively seems faster? (we are not doing second invalidation of the control on every keypress).

Changes which do mutate content may be susceptible to the same style of issue, though on web/`react-dom` in Chrome, this seems to not try to preserve selection at all if the selection is uncontrolled, so this seems like less of an issue. 

I haven't yet looked at old arch, but my guess is we have similar issues there, and could be fixed in similar ways (though, we've been trying to avoid changing it as much as possible, and 0.76+ has new arch as default, so not sure if worth fixing in old impl as well if this is very long running issue).

Changelog:
[iOS][Fixed] - Fix cursor moving in iOS controlled single line TextInput on Autocorrection (New Arch)

Differential Revision: D64121570
NickGerleman added a commit to NickGerleman/react-native that referenced this issue Oct 15, 2024
…riggered in controlled single line TextInput on iOS (New Arch) (facebook#46970)

Summary:

Fixes facebook#44157

This one is a bit of a doozy...

During auto-correct in UITextField (used for single line TextInput) iOS will mutate the buffer in two parts, non-atomically. After the first part, after iOS triggers `textFieldDidChange`, selection is in the wrong position. If we set full new AttributedText at this point, we propagate the incorrect cursor position, and it is never restored.

In the common case, where we are not mutating text in the controlled component, we shouldn't need to be setting AttributedString in the first place, and we do have an equality comparison there currently. But it is defeated because attributes are not identical. There are a few sources of that:
1. NSParagraphStyle is present in backing input, but not the AttributedString we are setting. 
2. Backing text has an NSShadow with no color (does not render) not in the AttributedText
3. Event emitter attributes change on each update, and new text does not inherit the attributes.

The first two are part of the backing input `typingAttributes`, even if we set a dictionary without them. To solve for them, we make attribute comparison insensitive to the attribute values in a default initialized control. There is code around here fully falling back to attribute insensitive comparison, which we would ideally fix to instead role into this "effective" attribute comparison.

The event emitter attributes being misaligned is a real problem. We fix in a couple ways.
1. We treat the attribute values as equal if the backing event emitter is the same
2. We set paragraph level event emitter as a default attribute so the first typed character receives it

After these fixes, scenario in facebook/react-native-website#4247 no longer repros in new arch. Typing in debug build also subjectively seems faster? (we are not doing second invalidation of the control on every keypress).

Changes which do mutate content may be susceptible to the same style of issue, though on web/`react-dom` in Chrome, this seems to not try to preserve selection at all if the selection is uncontrolled, so this seems like less of an issue. 

I haven't yet looked at old arch, but my guess is we have similar issues there, and could be fixed in similar ways (though, we've been trying to avoid changing it as much as possible, and 0.76+ has new arch as default, so not sure if worth fixing in old impl as well if this is very long running issue).

Changelog:
[iOS][Fixed] - Fix cursor moving in iOS controlled single line TextInput on Autocorrection (New Arch)

Reviewed By: philIip

Differential Revision: D64121570
NickGerleman added a commit to NickGerleman/react-native that referenced this issue Oct 15, 2024
…n controlled single line TextInput on iOS (New Arch) (facebook#46970)

Summary:

Fixes facebook#44157

This one is a bit of a doozy...

During auto-correct in UITextField (used for single line TextInput) iOS will mutate the buffer in two parts, non-atomically. After the first part, after iOS triggers `textFieldDidChange`, selection is in the wrong position. If we set full new AttributedText at this point, we propagate the incorrect cursor position, and it is never restored.

In the common case, where we are not mutating text in the controlled component, we shouldn't need to be setting AttributedString in the first place, and we do have an equality comparison there currently. But it is defeated because attributes are not identical. There are a few sources of that:
1. NSParagraphStyle is present in backing input, but not the AttributedString we are setting. 
2. Backing text has an NSShadow with no color (does not render) not in the AttributedText
3. Event emitter attributes change on each update, and new text does not inherit the attributes.

The first two are part of the backing input `typingAttributes`, even if we set a dictionary without them. To solve for them, we make attribute comparison insensitive to the attribute values in a default initialized control. There is code around here fully falling back to attribute insensitive comparison, which we would ideally fix to instead role into this "effective" attribute comparison.

The event emitter attributes being misaligned is a real problem. We fix in a couple ways.
1. We treat the attribute values as equal if the backing event emitter is the same
2. We set paragraph level event emitter as a default attribute so the first typed character receives it

After these fixes, scenario in facebook/react-native-website#4247 no longer repros in new arch. Typing in debug build also subjectively seems faster? (we are not doing second invalidation of the control on every keypress).

Changes which do mutate content may be susceptible to the same style of issue, though on web/`react-dom` in Chrome, this seems to not try to preserve selection at all if the selection is uncontrolled, so this seems like less of an issue. 

I haven't yet looked at old arch, but my guess is we have similar issues there, and could be fixed in similar ways (though, we've been trying to avoid changing it as much as possible, and 0.76+ has new arch as default, so not sure if worth fixing in old impl as well if this is very long running issue).

Changelog:
[iOS][Fixed] - Fix cursor moving in iOS controlled single line TextInput on Autocorrection (New Arch)

Reviewed By: philIip

Differential Revision: D64121570
NickGerleman added a commit to NickGerleman/react-native that referenced this issue Oct 15, 2024
…n controlled single line TextInput on iOS (New Arch) (facebook#46970)

Summary:

Fixes facebook#44157

This one is a bit of a doozy...

During auto-correct in UITextField (used for single line TextInput) iOS will mutate the buffer in two parts, non-atomically. After the first part, after iOS triggers `textFieldDidChange`, selection is in the wrong position. If we set full new AttributedText at this point, we propagate the incorrect cursor position, and it is never restored.

In the common case, where we are not mutating text in the controlled component, we shouldn't need to be setting AttributedString in the first place, and we do have an equality comparison there currently. But it is defeated because attributes are not identical. There are a few sources of that:
1. NSParagraphStyle is present in backing input, but not the AttributedString we are setting. 
2. Backing text has an NSShadow with no color (does not render) not in the AttributedText
3. Event emitter attributes change on each update, and new text does not inherit the attributes.

The first two are part of the backing input `typingAttributes`, even if we set a dictionary without them. To solve for them, we make attribute comparison insensitive to the attribute values in a default initialized control. There is code around here fully falling back to attribute insensitive comparison, which we would ideally fix to instead role into this "effective" attribute comparison.

The event emitter attributes being misaligned is a real problem. We fix in a couple ways.
1. We treat the attribute values as equal if the backing event emitter is the same
2. We set paragraph level event emitter as a default attribute so the first typed character receives it

After these fixes, scenario in facebook/react-native-website#4247 no longer repros in new arch. Typing in debug build also subjectively seems faster? (we are not doing second invalidation of the control on every keypress).

Changes which do mutate content may be susceptible to the same style of issue, though on web/`react-dom` in Chrome, this seems to not try to preserve selection at all if the selection is uncontrolled, so this seems like less of an issue. 

I haven't yet looked at old arch, but my guess is we have similar issues there, and could be fixed in similar ways (though, we've been trying to avoid changing it as much as possible, and 0.76+ has new arch as default, so not sure if worth fixing in old impl as well if this is very long running issue).

Changelog:
[iOS][Fixed] - Fix cursor moving in iOS controlled single line TextInput on Autocorrection (New Arch)

Reviewed By: javache, philIip

Differential Revision: D64121570
@cortinico cortinico added the Resolution: Fixed A PR that fixes this issue has been merged. label Oct 16, 2024
blakef pushed a commit that referenced this issue Nov 12, 2024
…n controlled single line TextInput on iOS (New Arch) (#46970)

Summary:
Pull Request resolved: #46970

Fixes #44157

This one is a bit of a doozy...

During auto-correct in UITextField (used for single line TextInput) iOS will mutate the buffer in two parts, non-atomically. After the first part, after iOS triggers `textFieldDidChange`, selection is in the wrong position. If we set full new AttributedText at this point, we propagate the incorrect cursor position, and it is never restored.

In the common case, where we are not mutating text in the controlled component, we shouldn't need to be setting AttributedString in the first place, and we do have an equality comparison there currently. But it is defeated because attributes are not identical. There are a few sources of that:
1. NSParagraphStyle is present in backing input, but not the AttributedString we are setting.
2. Backing text has an NSShadow with no color (does not render) not in the AttributedText
3. Event emitter attributes change on each update, and new text does not inherit the attributes.

The first two are part of the backing input `typingAttributes`, even if we set a dictionary without them. To solve for them, we make attribute comparison insensitive to the attribute values in a default initialized control. There is code around here fully falling back to attribute insensitive comparison, which we would ideally fix to instead role into this "effective" attribute comparison.

The event emitter attributes being misaligned is a real problem. We fix in a couple ways.
1. We treat the attribute values as equal if the backing event emitter is the same
2. We set paragraph level event emitter as a default attribute so the first typed character receives it

After these fixes, scenario in facebook/react-native-website#4247 no longer repros in new arch. Typing in debug build also subjectively seems faster? (we are not doing second invalidation of the control on every keypress).

Changes which do mutate content may be susceptible to the same style of issue, though on web/`react-dom` in Chrome, this seems to not try to preserve selection at all if the selection is uncontrolled, so this seems like less of an issue.

I haven't yet looked at old arch, but my guess is we have similar issues there, and could be fixed in similar ways (though, we've been trying to avoid changing it as much as possible, and 0.76+ has new arch as default, so not sure if worth fixing in old impl as well if this is very long running issue).

Changelog:
[iOS][Fixed] - Fix cursor moving in iOS controlled single line TextInput on Autocorrection (New Arch)

Reviewed By: javache, philIip

Differential Revision: D64121570

fbshipit-source-id: 2b3bd8a3002c33b68af60ffabeffe01e25c7ccfe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: TextInput Related to the TextInput component. Issue: Author Provided Repro This issue can be reproduced in Snack or an attached project. Newer Patch Available Resolution: Fixed A PR that fixes this issue has been merged.
Projects
None yet
Development

No branches or pull requests

10 participants