Skip to content

Latest commit

 

History

History
626 lines (545 loc) · 28.2 KB

Appendix.md

File metadata and controls

626 lines (545 loc) · 28.2 KB
Raw

Appendix

This appendix supplements the definition, the missed data and description of paper.

Table 1 The range of tags of upstream version

This table specify the merge points of upstream Android tags in corresponding versions(branches).

Project Downstream Version Upstream(AOSP) Version Merge Commits
Version #File #Loc #Commit Version #File #Loc #Commit #MerCmt #CflCmt
AOSPA Sapphire 27,539 4,575,525 674,953 android12-gsi(898ad0236f7) 27,308 4,526,922 611,807 46 37
Ruby 26,497 4,165,240 501,248 android11-qpr1-d-release(ca05b4c5f77) 26,429 4,147,331 650,983 64 47
Quartz 19,939 3,188,444 420,479 android11-d2-release(823838e9efc) 19,913 3,175,862 498,028 99 52
CalyxOS android12 27,384 4,530,132 651,640 android12-qpr1-d-release(android-12.0.0_r29) 27,308 4,526,923 649,754 5 2
android11 26,699 4,169,261 482,229 android11-qpr3-release(android-11.0.0_r46) 26,463 4,155,454 497,949 10 1
LineageOS LineageOS-19.1 27,685 4,576,171 663,065 android12-qpr3-s1-release(android-12.1.0_r7) 27,469 4,558,550 659,793 3 1
LineageOS-18.1 26,713 4,190,090 499,427 android11-gsi(android-11.0.0_r46) 22,534 3,669,603 505,590 17 8
LineageOS-17.1 22,951 3,711,094 426,543 android-s-beta-4(android-10.0.0_r41) 22,455 3,652,993 571,185 28 12
LineageOS-16.0 2,0293 3,212,780 371,425 android-s-beta-4(android-9.0.0_r61) 19,871 3,171,811 369,065 50 12
OmniROM android-12.0 27,351 4,530,005 650,684 android12-gsi(android-12.0.0_r28) 27,308 4,526,922 650,983 4 2
android-11 25,866 4,147,440 502,111 android12-gsi(android-11.0.0_r38) 25,796 4,131,630 687,952 178 106
android-10 22,744 3,711,094 426,463 android12-gsi(android-10.0.0_r41) 22,558 3,677,983 650,983 10 3
android-9 19,930 3,186,526 371,214 android10-qpr2-release(android-9.0.0_r34) 19,824 3,173,299 423,947 43 3
IndustrialX S 35,546 5,866,790 661,243 android12-gsi(android-12.0.0_r10) 27,161 4,511,991 648,567 2 2
R 31,181 4,909,996 491,418 android12-gsi(android-11.0.0_r35) 26,454 4,152,622 418,833 14 14

Table 2 The "older versions" specification(Review C Q1(2))

Q: how are "older upstream versions" chosen?

The commits in older upstream versions means these commits are not in the current latest merge points (i.e. android-12.1.0_r7 at android12-qpr3-s1-release for LineageOS-19.1 ) but in the whole commit history of upstream Android.

Name Description
$Commits_U$ All commits in the current latest merge points in current branch/version (i.e. android-12.1.0_r7 at android12-qpr3-s1-release for LineageOS-19.1 ) of upstream Android.
$Commits_D$ All commits in the latest merge points in current branch/version (i.e. LineageOS-19.1) of downstream projects.
$Commits_O$ All commits in the upstream commit history but not in $Commits_U$
$U_e$ The intersection of the commit set of the current entity and $Commits_U$
$D_e$ The intersection of the commit set of the current entity and $Commits_D$
$O_e$ The intersection of the commit set of the current entity and $Commits_O$

Table 3 The combinations for enitity's ownership identification Parper P3 (Review C Q1(4) and Review A “Combination of Ue, De, and Oe”)

Q: how does it deal with situations when (Ue!=0, De!=0, Oe!=0) or (Ue==0, De!=0, Oe!=0)?

There are $2^3 = 8$ combinations for Ue, De, and Oe in sum，and here are the other 4 combinations we missed to present in paper.

Ue De Oe ownership
$\neq\emptyset$ $=\emptyset$ $=\emptyset$ “actively native”
$=\emptyset$ $=\emptyset$ $\neq\emptyset$ "obsoletely native"
$\neq\emptyset$ $\neq\emptyset$ $=\emptyset$ "intrusively native"
$=\emptyset$ $\neq\emptyset$ $=\emptyset$ "extensive"
$=\emptyset$ $=\emptyset$ $=\emptyset$ "IMPOSSIBLE"
$=\emptyset$ $\neq\emptyset$ $\neq\emptyset$ "intrusive native"
$\neq\emptyset$ $\neq\emptyset$ $\neq\emptyset$ "intrusive native"
$\neq\emptyset$ $=\emptyset$ $\neq\emptyset$ "obsoletely native"

Table 4 The formulaic definition of concepts (Review B)

Q: . There is no formal definition that systematically describes the DepFCD model. It is better to clearly give out the definition of the dependency graph, updated graph, dependency facade, conflicts, etc.

The formal definition of the dependency graph, updated graph, dependency facade, conflicts, etc.

concept description definition
vertex set The set of entities(vertexes) of dependency graph $V = \lbrace {e}\rbrace$
dependency set The set of edges(dependencies) of dependency graph $D = \lbrace {d = e_i \to e_j}\rbrace$
dependency graph The initial dependency graph provided by ENRE $G = \langle V, D\rangle$
entity's restriction level The non-SDK restriction level of entities imposed by Android ${e.restrictlevel\in\lbrace’sdk’, 'unsupported’, 'max-target-x', 'blocked'\rbrace}$
entity's operation The operation downstream made to upstream entities ${e, e.operation\in\lbrace'modify\ \ the\ \ modifier', 'modify\ \ the\ \ annotation', 'modify\ \ the\ \ parent\ class', etc\rbrace}$
entity's ownership The ownership of entities ${e, e.ownership\in\lbrace'actively\ native', 'obsoletely\ native', 'intrusively\ native', 'extensive'\rbrace}$
updated graph The updated dependency graph, which included restriction level, operation, ownership $G' = \langle V', D\rangle, V' = \lbrace{e, e.ownership\neq\emptyset, e.operation\neq\emptyset, e.restrict_level\neq\emptyset \rbrace}$
dependency facade The dependency facade which downstream coupling with upstream $G_F = \langle V', D'\rangle \ D' = \lbrace {d = e_i \to e_j, \ e_i.ownership == 'extensive'\ \land \ e_j.ownership.endwith('native') \cup \ e_j.ownership == 'extensive'\ \land \ e_i.ownership.endwith('native') \rbrace }$
conflicts The entities which conflicts take place ${e, e.conflict==CONFLICT\ BLOCK}$

Table 5 The description of metrics (Review B)

Q: the definitions in section IV-A and the metrics Pact, Pins, Pext, Pobs, PE, and PD are all given without a description name and are hard to understand.

The description name of the metrics Pact, Pins, Pext, Pobs, PE, and PD

metrics description
$P_{act}$ the proportion of actively native entities in a downstream dependency graph $G^′_D.$
$P_{ins}$ the proportion of intrusively native entities in a downstream dependency graph $G^′_D.$
$P_{ext}$ the proportion of extensive entities in a downstream dependency graph $G^′_D.$
$P_{obs}$ the proportion of obsoletely native entities in a downstream dependency graph $G^′_D.$
$PE$ the proportion of entities $E_F$ in $G_F$ to all of the entities $E$ in $G^′_D.$
$PD$ the proportion of dependencies $D_F$ in $G_F$ to all of the dependencies $D$ in $G^′_D.$

Table 6 The mapping between "Cases" and "Features" in Table VIII (Review C Q2, Q3)

Q2:please clarify the mapping between the "Cases" column and the "Features" column in Table VIII?

Q3: please provide statistics (e.g., frequency counts) for the dependency facade in the "Features" column?

There are 3 kinds of mappings between the “Cases” and “Features” in Table VIII. Each “Case” can be characterized by several “Features”.

  • Interface-level dependency (highlighted in yellow): According to dependency definitions in Table II, each case can be mapped into several of 7 features, including “Call”, “Aggregate”, “Inherit”, etc.
  • Dependency constraints (highlighted in blue): According to the entities of non-SDK restriction annotations, each case can be mapped into the corresponding dependency constraint.
  • Intrusion operations (highlighted in purple): According to intrusion operation definitions in Table IV, each case can be mapped into the feature such as ⑩-“modify local variable” and ⑪-“modify method body"
Conflict Location The Cases Regarding Downstream Modifications Features Frequency counts
Method Block c ⑩ 24
a,b,d,e ⑪ 108
Class Field a,b ⑧ 31
Import a,b,c,d ④ 31
a,b ⑤ 7
a,b ⑥ 4
Parameters a,b ⑨ 18
Accessibility a,b,c ① 10
Extensive method b ⑤ 1
a ⑥ 3
c ⑮ 1
Final a ② 2
a ⑧ 2
Annotation a,b ③ 2
Inner-classes / ⑦ 1

The "Others" in Table VIII:

  1. “others” in Method blocks means the conflicts occur at the comment or javadoc;

  2. “others” in Import means the conflicts take place due to importing built-in package or class;

  3. “others” in Extensive method means downstream add extensive method in native file.

The 1. and 2. are not relative to dependency facade, 3. will lead to dependency edges from extensive to native entity, which is part of dependency facade.

“49+16” in Table VIII:

  • These conflicts（16） are due to git text matching errors and the conflict entity is not the same position as the place that actually changed；
  • Meanwhile, there were some conflicts（49） in the Android itself during the upgrade process, which were repeated during the simulation, so these conflicts were classified as Upstream Update.

Table 7 The comparison of percentage of dependencies from intrusively native to native (Review C Q1(3))

C Q1(3):why is an "extensive" entity always required for detecting a dependency facade, say intrusive modifications?

When an entity is identified as "intrusively native", dependencies in it may originate from the upstream or created by the downstream. We calculated the percentage of dependencies between intrusively native and native which upstream or downstream imposed.

project the percentage of dependencies between intrusively native and native which upstream imposed the percentage of dependencies between intrusively native and native which downstream imposed
AOSPA-Sapphire 98.55% 1.45%
AOSPA-ruby-staging 99.26% 0.74%
AOSPA-quartz 99.15% 0.85%
Calyxos-12 99.45% 0.55%
Calyxos-11 99.35% 0.65%
lineageOS-19.1 99.12% 0.88%
lineageOS-18.1 98.81% 1.19%
lineageOS-17.1 99.03% 0.97%
lineageOS-16.0 99.01% 0.99%
OmniROM-12.0 99.38% 0.62%
OmniROM-11 99.03% 0.97%
OmniROM-10 98.87% 1.13%
OmniROM-9 98.99% 1.01%
Industrial-S 97.55% 2.45%
Industrial-R 97.80% 2.20%
AVERAGE 98.38% 1.62%

According to table 7, we can see 98.38% dependencies between intrusively native and native are originated from the upstream, which should not be included in the dependency facade. And for the left 1.62% dependencies, we will study them in the next step.

Table 8 The relation of two hotspots which RQ2 identified with conflicts we collected (Review C)

**Q: RQ2 also identifies two hotspots of intrusive modifications; it'd be more comprehensive if more statistics about all (or most) of the intrusive entities before giving the two hotspots. Also, may discuss more: what does it imply for an entity to be a hotspot? **

hotspot 1 : com.android.server.pm.PackageManagerService

hotspot 2 : android.provider.Settings

All versions of downstream projects have modified these 2 files. Thus, we count the conflict commits related to these 2 hotspots and find nearly 10% conflicts related to them.

project P(hotspot_1 / #CflCmt) P(hotspot_2 / #CflCmt)
AOSPA-Sapphire 7/37 2/37
AOSPA-Ruby 4/47 2/47
AOSPA-Quartz 2/52 6/52
CalyxOS-12 0/2 0/2
CalyxOS-11 0/1 0/1
LineageOS-19.1 0/1 0/1
LineageOS-18.1 1/8 0/8
LineageOS-17.1 1/12 1/12
LineageOS-16.0 0/12 0/12
OmniROM-12 1/2 0/2
OmniROM-11 7/106 8/106
OmniROM-10 0/3 0/3
OmniROM-9 0/3 1/3
IndustrialX-S 2/2 1/2
IndustrialX-R 10/14 1/14
AVERAGE 9.45%
SUMMARY 35/302=11.6% 22/302=7.3%

Table 9 The top 2% conflict files in all conflicts we collected(Review C)

**Q: RQ2 also identifies two hotspots of intrusive modifications; it'd be more comprehensive if more statistics about all (or most) of the intrusive entities before giving the two hotspots. Also, may discuss more: what does it imply for an entity to be a hotspot? **

We counted the times each file conflicts and these 2 hotspots are in the top 2% conflict files. The hotspots indicates that the downstream versions commonly customize these classes and indeed incur merge conflicts.

Files Conflict times
services/core/java/com/android/server/ConnectivityService.java 56
services/core/java/com/android/server/am/ActivityManagerService.java 55
telephony/java/android/telephony/CarrierConfigManager.java 50
services/core/java/com/android/server/audio/AudioService.java 44
services/core/java/com/android/server/wm/ActivityRecord.java 40
packages/SystemUI/src/com/android/systemui/statusbar/policy/MobileSignalController.java 39
services/core/java/com/android/server/BluetoothManagerService.java 36
==services/core/java/com/android/server/pm/PackageManagerService.java== 35
services/core/java/com/android/server/wm/ActivityStack.java 35
packages/SettingsLib/src/com/android/settingslib/bluetooth/CachedBluetoothDevice.java 29
services/core/java/com/android/server/am/ActiveServices.java 27
services/core/java/com/android/server/locksettings/LockSettingsService.java 23
wifi/java/android/net/wifi/WifiConfiguration.java 23
services/core/java/com/android/server/audio/AudioDeviceBroker.java 23
core/java/android/bluetooth/BluetoothAdapter.java 23
services/core/java/com/android/server/Watchdog.java 23
==core/java/android/provider/Settings.java== 22
services/core/java/com/android/server/wm/WindowManagerService.java 21
services/core/java/com/android/server/audio/BtHelper.java 21
packages/SystemUI/src/com/android/systemui/statusbar/policy/WifiSignalController.java 21

Table 10 The manual checked results of our approach (Review A Q1)

A Q1: The classification of native entities, especially the "obsoletely native entity".

We conducted the manual check on the results of our approach for all projects we collected, and in all entities we selected randomly, 99.94% of them are true to life.

project The number of entities manual checked The number of wrong-identified entities manual checked accuracy
AOSPA-Sapphire 6747 3 99.96%
AOSPA-ruby 942 0 100.00%
AOSPA-quartz 720 0 100.00%
CalyxOS-12 229 4 98.25%
CalyxOS-11 440 0 100.00%
lineage-19.1 1029 0 100.00%
lineage-18.1 1269 0 100.00%
lineage-17.1 1322 0 100.00%
lineage-16.0 1079 0 100.00%
OmniROM-12 278 0 100.00%
OmniROM-11 1786 4 99.78%
OmniROM-10 1387 0 100.00%
OmniROM-9 1124 0 100.00%
SUMMARY 18352 11 99.94%

Table 11 The rebase and cherry-pick operations in downstream projects.(Review C Q4)

Q: please comment on how conflicts caused by rebase, cherry-pick, etc. may be identified and related to the dependency facade?

In the open-source projects:

  • For rebase operation, according to [1], even though the rebasing branches rewrites the evolutionary history and the previous commits are missing after rebasing, it could be restored by identifying the force-pushed event that happened to the head branch of one pull request and then retrieving the missing commits. However, we do not find any force-pushed event in the downstream GitHub repos, which mean the downstream do not conduct "rebase".

  • For cherry-pick operation, which could be restored by identifying the "cherry picked from commit ” in commit history[2], we collected all commits related to "cherry-pick" in the downstream commit history, which includes both upstream and downstream "cherry-pick". However, we only identified 8 cherry-picks combined with conflicts and they are all conducted by upstream.

Project Commits number related to cherry-pick Conflicts related to cherry-pick
AOSPA 7484 8
CalyxOS 6773 8
LineageOS 6749 8
OmniROM 6725 8

In the industrial projects:

Even though the "rebase" and "cherry-pick" commands are used during the development, "rebase" is hardly used and "cherry-pick" is used to synchronize a small number of problems in the downstream repo, the main problem is the conflicts caused by "merge".

[1]T. Ji, L. Chen, X. Yi and X. Mao, "Understanding Merge Conflicts and Resolutions in Git Rebases," 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), 2020, pp. 70-80, doi: 10.1109/ISSRE5003.2020.00016.

[2]Panuchart Bunyakiati and Chadarat Phipathananunth. 2017. Cherry-picking of code commits in long-running, multi-release software. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 994–998. https://doi.org/10.1145/3106237.3122818