Adapt iOS16+ dictation

[hellohublot]

Upstream PR Link

facebook#37188

Summary:

facebook#19687
https://developer.apple.com/forums/thread/711413

When system version is lower than iOS 16, it does not support dictation and keyboard working at the same time, so if we modify the text, the system will immediately interrupt the dictation, so we need to prohibit modification of the text during recording and recognition

When system version is higher than iOS 16, dictation and keyboard can work at the same time, so textInputMode.primaryLanguage is no longer changed to dictation, so we can modify the text during recording, because the system will not interrupt, but we cannot modify the text during recognition, Because the system will temporarily add a _UITextPlaceholderAttachment to display the recognition UIActivityIndicator

Changelog:

[IOS] [FIXED] - Adapt iOS16+ dictation judge condition

Test Plan:

Test Code

constructor(props) {
  super(props)
  this.state = {
    value: '',
    logList: [],
  }
}

render() {
  return (
    <View style={{ flex: 1, padding: 50, backgroundColor: 'white'}}>
      <TextInput 
        style={{ marginTop: 20, height: 50, borderWidth: 1, borderColor: 'black' }} 
        placeholder={'please input'}
        placeholderTextColor={'gray'}
        value={this.state.value}
        onChangeText={value => {
          let logList = this.state.logList
          logList.push(value.length <= 0 ? 'null' : value.replace(/\uFFFC/g, '_uFFFC'))
          this.setState({ value, logList })
        }}
        onEndEditing={() => this.setState({ value: '', logList: [] })}
      />
      <FlatList
        style={{ marginTop: 20, maxHeight: 300, borderWidth: 1, borderColor: 'black' }}
        data={this.state.logList}
        renderItem={({item}) => (
            <Text>{item}</Text>
        )}
      />
    </View>
  )
}

Case A: Required < iOS16

1. ensure that iOS Dictation TextInput ends abruptly between words facebook/react-native#18890 can work well and dictation will not be interrupted immediately

_case_a.mp4

Case B: Required >= iOS16

1. ensure that iOS Dictation TextInput ends abruptly between words facebook/react-native#18890 can work well and dictation will not be interrupted immediately

_case_b.mp4

Case C: Required >= iOS16

1. start dictation
2. then do not speak any words
3. then end dictation
4. verify that onChangeText will callback "\uFFFC" once
5. and then verify onChangeText callback an empty string "" once

_case_c.MP4

Case D: Required >= iOS16

1. start dictation
2. input some text while speaking some words
3. then end dictation
4. and verify that the onChangeText callback work fine.

_case_d.MP4

Case E: Required >= iOS16

1. start dictation
2. say a word
3. and then switch the keyboard to other language
4. verify that dictation will not end
5. continue say some word
6. verify the onChangeText callback work fine.

_case_e.MP4

[hellohublot]

@flodnv Thanks, It's all done, I just cherry-pick the (facebook@e8b4bb0)

[flodnv]

Thanks!

[flodnv]

@hellohublot any idea why the GH action is failing? 😕

[hellohublot]

Thanks, I didn't modify any android code, but I can give you some clues

According to https://github.com/Expensify/react-native/actions/runs/5080321413/jobs/9137029909, the CI error message is

/app/sdks/hermes/API/hermes/hermes.cpp:639:32: error: non-virtual member function marked 'override' hides virtual member functions
         const jsi::Value &value) override;
                                  ^
/app/ReactCommon/jsi/jsi/jsi.h:340:3: note: hidden overloaded virtual function 'facebook::jsi::Runtime::setPropertyValue' declared here: type mismatch at 1st parameter ('facebook::jsi::Object &' vs 'const jsi::Object &')

The reason for the error here is because facebook::jsi::Object should be const facebook::jsi::Object

So we can integrate this PR facebook@03b17d9#diff-4d691d151cba9dc2f79cb3ae582fc99ab8d639dda355d367aef25fc136ad0f93R340, but I guess that will also take some time, so we can merge our PR first, because It doesn't modify any android code

[flodnv]

I'm a bit confused about your proposal, won't that result in breaking the GH action for everyone?

Additionally you seem to suggest your PR did not break it? However, I see a more recent PR (example) that is working 😕

[hellohublot]

@flodnv Thanks.
https://github.com/Expensify/react-native/actions/workflows/test-docker-android.yml
From here, we can see that our Test Docker Android Image has not been executed successfully for several months, which coincides with the submission time of the PR I suggested to integrate.

This PR https://github.com/Expensify/react-native/pull/57/checks didn't run Test Docker Android Image, Because the target branch that these PRs want to merge is not Expensify-react-native:main, pull_request.branches.main in test-docker-android.yml cannot be triggered, so they will not execute Test Docker Android Image

[flodnv]

Hmmm thanks for the details; this is confusing, I've posted about it here: https://expensify.slack.com/archives/C01GTK53T8Q/p1685557998465289

[hellohublot]

@flodnv Thanks, I have switched branch to ExpensifyRC1-0.72.0-alpha.0, #49 (review) because it seems like only this branch will be released.

[flodnv]

Thank you!