Support delta installer for updates on Windows #9565

simonhong · 2020-05-01T03:36:52Z

Add parameter to create_dist command for generating delta installer

Let's start investigation about delta update with below archive type.
Maybe INCREMENTAL_ARCHIVE_TYPE is used for delta installer.

enum ArchiveType {
  UNKNOWN_ARCHIVE_TYPE,     // Unknown or uninitialized.
  FULL_ARCHIVE_TYPE,        // Full chrome.7z archive.
  INCREMENTAL_ARCHIVE_TYPE  // Incremental or differential archive.
};

Currently, mini_installer target uses courgette algorithm for generating delta installer.
Courgette: https://www.chromium.org/developers/design-documents/software-updates-courgette

According to the below code in chrome/installer/setup/setup_main.cc, maybe setup.exe generates output file(newer version file) by using patch file(new delta file?) and input file(current version). and output file would be a complete file for update to next version.

 } else if (cmd_line.HasSwitch(installer::switches::kPatch)) {
    const std::string patch_type_str(
        cmd_line.GetSwitchValueASCII(installer::switches::kPatch));
    const base::FilePath input_file(
        cmd_line.GetSwitchValuePath(installer::switches::kInputFile));
    const base::FilePath patch_file(
        cmd_line.GetSwitchValuePath(installer::switches::kPatchFile));
    const base::FilePath output_file(
        cmd_line.GetSwitchValuePath(installer::switches::kOutputFile));

    if (patch_type_str == installer::kCourgette) {
      *exit_code = installer::CourgettePatchFiles(input_file,
                                                  patch_file,
                                                  output_file);
    } else if (patch_type_str == installer::kBsdiff) {
      *exit_code = installer::BsdiffPatchFiles(input_file,
                                               patch_file,
                                               output_file);
#if BUILDFLAG(ZUCCHINI)
    } else if (patch_type_str == installer::kZucchini) {
      *exit_code =
          installer::ZucchiniPatchFiles(input_file, patch_file, output_file);
#endif  // BUILDFLAG(ZUCCHINI)
    } else {
      *exit_code = installer::PATCH_INVALID_ARGUMENTS;
    }
  }

create_installer_archive.py generates patch file by mini_installer target.
Uncompressed chrome.7z file is used to get diff.

template("generate_mini_installer")

The text was updated successfully, but these errors were encountered:

simonhong · 2020-05-07T02:24:03Z

Got strange error while generating delta installer.
Previous version's chrome.7z extracting is failed. but extracted properly when I do it in console.
Curious why it think as an overwrite??? - Before running GetPrevVersion, this script delete previous temp dir. So, it should not be an overwrite when extracting.

=> Found the reason
when below temp_dir is passed with -o'"temp_dir"', temp_dir is generated under C:/.
However, temp_dir is generated in cwd when it's passed with -o'temp_dir'.
Because of this, it warns about overwriting. Normally, that temp_dir becomes empty because MakeStagingDirectory deletes it before extracting it into temp_diir.

  cmd = [lzma_exec,
         'x',
         '-o%s' % temp_dir,
         prev_archive_file,
         'Chrome-bin/*/chrome.dll',]

[1/279] ACTION //components/resources:about_credits(//build/toolchain/win:win_clang_x64)
[2/9] ACTION //chrome/installer/mini_installer:mini_installer_archive(//build/toolchain/win:win_clang_x64)
FAILED: chrome.7z setup.ex_ gen/chrome/installer/mini_installer/mini_installer/packed_files.rc chrome.packed.7z
C:/Projects/brave/brave-browser/vendor/depot_tools/bootstrap-3_8_0_chromium_8_bin/python/bin/python.exe ../../chrome/tools/build/win/create_installer_archive.py --build_dir . --staging_dir gen/chrome/installer/mini_installer/mini_installer --input_file ../../chrome/installer/mini_installer/chrome.release --resource_file_path gen/chrome/installer/mini_installer/mini_installer/packed_files.rc --target_arch=x64 --distribution=_brave --output_dir . --chrome_runtime_deps gen/chrome_component.runtime_deps --setup_runtime_deps gen/setup.runtime_deps --last_chrome_installer=C:/build/installer --setup_exe_format=DIFF --diff_algorithm=COURGETTE --verbose --skip_signing --enable_hidpi=1 --depfile gen/chrome/installer/mini_installer/archive.d
['../../chrome/tools/build/win/create_installer_archive.py', '--build_dir', '.', '--staging_dir', 'gen/chrome/installer/mini_installer/mini_installer', '--input_file', '../../chrome/installer/mini_installer/chrome.release', '--resource_file_path', 'gen/chrome/installer/mini_installer/mini_installer/packed_files.rc', '--target_arch=x64', '--distribution=_brave', '--output_dir', '.', '--chrome_runtime_deps', 'gen/chrome_component.runtime_deps', '--setup_runtime_deps', 'gen/setup.runtime_deps', '--last_chrome_installer=C:/build/installer', '--setup_exe_format=DIFF', '--diff_algorithm=COURGETTE', '--verbose', '--skip_signing', '--enable_hidpi=1', '--depfile', 'gen/chrome/installer/mini_installer/archive.d']
C:/build/installer\chrome.7z
gen/chrome/installer/mini_installer/mini_installer\temp_installer_archive
Running .\..\..\third_party\lzma_sdk\Executable\7za.exe x -o"gen/chrome/installer/mini_installer/mini_installer\temp_installer_archive" C:/build/installer\chrome.7z Chrome-bin\*\chrome.dll
Traceback (most recent call last):
  File "../../chrome/tools/build/win/create_installer_archive.py", line 665, in <module>
    sys.exit(main(options))
  File "../../chrome/tools/build/win/create_installer_archive.py", line 538, in main
    options.output_name)
  File "../../chrome/tools/build/win/create_installer_archive.py", line 188, in GetPrevVersion
    RunSystemCommand(cmd, options.verbose)
  File "../../chrome/tools/build/win/create_installer_archive.py", line 232, in RunSystemCommand
    (e.cmd, e.returncode, e.output))
Exception: Error while running cmd: ['.\\..\\..\\third_party\\lzma_sdk\\Executable\\7za.exe', 'x', '-o"gen/chrome/installer/mini_installer/mini_installer\\temp_installer_archive"', 'C:/build/installer\\chrome.7z', 'Chrome-bin\\*\\chrome.dll']
Exit code: 2
Command output:

7-Zip (a) [64] 15.14 : Copyright (c) 1999-2015 Igor Pavlov : 2015-12-31

Scanning the drive for archives:
1 file, 220406223 bytes (211 MiB)

Extracting archive: C:\build\installer\chrome.7z
--
Path = C:\build\installer\chrome.7z
Type = 7z
Physical Size = 220406223
Headers Size = 2965
Method = Copy
Solid = -
Blocks = 187


Would you like to replace the existing file:
  Path:     \gen/chrome/installer/mini_installer/mini_installer\temp_installer_archive\Chrome-bin\81.1.8.90\chrome.dll
  Size:     134749328 bytes (129 MiB)
  Modified: 2020-05-05 11:27:19
with the file from archive:
  Path:     Chrome-bin\81.1.8.90\chrome.dll
  Size:     134749328 bytes (129 MiB)
  Modified: 2020-05-05 11:27:19
? (Y)es / (N)o / (A)lways / (S)kip all / A(u)to rename all / (Q)uit?
ERROR: Can't allocate required memory!

Archives with Errors: 1


ninja: build stopped: subcommand failed.
null
null
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! brave@1.10.23 create_dist: `node ./scripts/commands.js create_dist "Release" "--skip_signing"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the brave@1.10.23 create_dist script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\simon\AppData\Roaming\npm-cache\_logs\2020-05-07T02_20_15_743Z-debug.log

mihaiplesa · 2020-05-11T12:37:53Z

Getting the following:

13:33:37  The variable "build_delta_installer" was set as a build argument
13:33:37  but never appeared in a declare_args() block in any buildfile.
13:33:37  
13:33:37  To view all possible args, run "gn args --list <out_dir>"
13:33:37  
13:33:37  The build continued as if that argument was unspecified.

simonhong · 2020-05-12T00:22:27Z

@mihaiplesa Did you build with both PRs (b-b and b-c) together?
build_delta_installer is declared in https://github.com/brave/brave-core/pull/5484/files#diff-a1d6e4460dc2ec6f82e6a22b56c1bbd8R37

simonhong · 2020-06-04T01:11:49Z

Set release-notes/exclude label because server-side stuff also should be ready for delta-installer.
This issue is only for client side support for delta installer.

mihaiplesa · 2020-07-16T16:52:14Z

Reopening since this should include Windows x86 which still fails as above.

mihaiplesa · 2022-01-17T21:58:13Z

Fresh log from a Windows x86 build with delta error below.

0.020614s\r\n[0117/161749.066:VERBOSE1:ensemble_create.cc(208)] Matched old(1,256182034,784896) to new(1,422865086,784896) --> 32\r\n[0117/161749.066:VERBOSE1:ensemble_create.cc(218)] done FindGenerators found 16 in 49.3365s\r\n[0117/161751.924:VERBOSE1:simple_delta.cc(35)] GenerateSimpleDelta 31 31\r\n[0117/161751.924:VERBOSE1:bsdiff_create.cc(96)] Start bsdiff\r\n[0117/161751.924:VERBOSE1:bsdiff_create.cc(123)] done divsufsort 0.000336\r\n[0117/161751.924:VERBOSE1:bsdiff_create.cc(338)] Control tuples: 1 copy bytes: 31 mistakes: 0 (skips: 1) extra bytes: 0\r\nUncompressed bsdiff patch size 34\r\nEnd bsdiff 0.000409\r\n[0117/161806.146:VERBOSE1:adjustment_method_2.cc(1238)] Adjuster::Adjust\r\n[0117/162037.342:VERBOSE1:adjustment_method_2.cc(1279)] Adjuster::Solve 148.357\r\n[0117/162751.365:VERBOSE1:adjustment_method_2.cc(1279)] Adjuster::Solve 432.799\r\n[0117/162754.396:VERBOSE1:label_manager.cc(34)] 667504 of 686005 labels pre-assigned.\r\n[0117/162754.397:VERBOSE1:label_manager.cc(55)] fill forward 18501\r\n[0117/162754.397:VERBOSE1:label_manager.cc(77)] fill backward 0\r\n[0117/162754.398:VERBOSE1:label_manager.cc(93)] infill 0\r\n[0117/162754.403:VERBOSE1:label_manager.cc(34)] 1517676 of 1543242 labels pre-assigned.\r\n[0117/162754.405:VERBOSE1:label_manager.cc(55)] fill forward 25566\r\n[0117/162754.406:VERBOSE1:label_manager.cc(77)] fill backward 0\r\n[0117/162754.408:VERBOSE1:label_manager.cc(93)] infill 0\r\n[0117/162759.792:ERROR:patch_generator_x86_32.h(92)] Cannot serialize encoded 'new' input. (old(1,18118014,166675456) => new(1,32,166675456))\r\n-gen failed.\r\n"

mihaiplesa · 2022-02-09T16:26:22Z

Apparently Windows x86 deltas got fixed by migrating to zucchini in brave/brave-core#12098

GeetaSarvadnya · 2022-02-16T15:04:00Z

Verification PASSED on

Brave | 1.35.102 Chromium: 98.0.4758.102 (Official Build) (64-bit)
-- | --
Revision | 273bf7ac8c909cde36982d27f66f3c70846a3718-refs/branch-heads/4758@{#1151}
OS | Windows 10 Version 21H2 (Build 19044.1526)

Verified the test plan from Fix windows delta updates brave-core#12098

x64 bit stub installers

Case 1: Release channel_x64bit_ Admin Privileges Yes_PASSED

Install stub installer BraveBrowserSetup.exe
Confirm latest release version 1.35.101 is pulled from the Omaha server
Upgrade the profile to test channel stub installer BraveBrowserSetup-64-r-test.exe
Confirm able to upgrade the profile to 1.35.102

Stub installer	Test channel

Case 2: Release channel_x64bit_ Admin Privileges No

ensured that BraveBrowserSetup-64-r-test.exe downloads/installs 1.35.102 Chromium: 98.0.4758.102 without any issues
ensured that upgrading from 1.35.101 Chromium: 98.0.4758.87 -> 1.35.102 Chromium: 98.0.4758.102 using BraveBrowserSetup-64-r-test.exe worked without any issues

Stub installer	Test channel

Case 3: Beta channel_x64bit_ Admin Privileges Yes_PASSED

Install brave beta standalone installer 1.36.90 BraveBrowserStandaloneBetaSetup.exe
Upgrade the profile to test channel BraveBrowserBetaSetup-64-be-test.exe
Confirm that the beta profile is upgraded to test channel 1.36.96

Stub installer	Test channel

Case 4: Beta channel_x64bit_ Admin Privileges NO

ensured that BraveBrowserBetaSetup-64-be-test.exe downloads/installs 1.36.96 Chromium: 98.0.4758.87 without any issues
ensured that upgrading from 1.36.94 Chromium: 98.0.4758.87 -> 1.36.96 Chromium: 98.0.4758.87 using BraveBrowserBetaSetup-64-be-test.exe worked without any issues

Stub installer	Test channel

Case 5: Nightly channel_x64bit_ Admin Privileges Yes_PASSED

Instal the nightly version 1.37.34 - BraveBrowserStandaloneNightlySetup.exe
Upgrade the profile to test channel BraveBrowserBetaSetup-64-ni-test.exe
Confirm that the nightly profile is upgraded to test channel 1.37.53

Stub installer	Test channel

Case 6: Nightly channel_x64bit_ Admin Privileges NO

ensured that BraveBrowserNightlySetup-64-ni-test.exe downloads/installs 1.37.53 Chromium: 98.0.4758.102 without any issues
ensured that upgrading from 1.37.52 Chromium: 98.0.4758.102 -> 1.37.53 Chromium: 98.0.4758.102 using BraveBrowserNightlySetup-64-ni-test.exe worked without any issues

Stub installer	Test channel

x32 bit stub installers

Case 1: Release channel_x32bit_ Admin Privileges Yes_PASSED

Install stub installer BraveBrowserSetup32.exe
Confirm latest release version 1.35.101 is pulled from the Omaha server
Upgrade the profile to test channel stub installer BraveBrowserSetup32-86-r-test`
Confirm able to upgrade the profile to test channel 1.35.102

Stub installer	Test channel

Case 2: Release channel_x32bit_ Admin Privileges No

ensured that BraveBrowserSetup32-86-r-test.exe downloads/installs 1.35.102 Chromium: 98.0.4758.102 without any issues
ensured that upgrading from 1.35.101 Chromium: 98.0.4758.87 -> 1.35.102 Chromium: 98.0.4758.102 using BraveBrowserSetup-64-r-test.exe worked without any issues

Stub installer	Test channel

Case 3: Beta channel_x32bit_ Admin Privileges Yes_PASSED

Install brave beta 1.36.90 BraveBrowserStandaloneBetaSetup32.exe
Upgrade the profile to test channel BraveBrowserBetaSetup32-86-be-test
Confirm profile is upgraded to test channel 1.36.94

Stub installer	Test channel

Case 4: Beta channel_x32bit_ Admin Privileges NO

ensured that BraveBrowserBetaSetup32-86-be-test.exe downloads/installs 1.36.94 Chromium: 98.0.4758.87 without any issues
ensured that upgrading from 1.36.90 Chromium: 98.0.4758.87 -> 1.36.94 Chromium: 98.0.4758.87 using BraveBrowserBetaSetup32-86-be-test.exe worked without any issues

Stub installer	Test channel

Case 5: Nightly channel_x32bit_ Admin Privileges Yes_PASSED

Install nightly build 1.37.33 BraveBrowserStandaloneNightlySetup32.exe
Upgrade the profile to test channel BraveBrowserNightlySetup32-86-ni-test
Confirmed that profile is upgraded to test channel 1.37.53

Stub installer	Test channel

Case 6: Nightly channel_32bit_ Admin Privileges NO

ensured that BraveBrowserNightlySetup32-86-ni-test.exe downloads/installs 1.37.53 Chromium: 98.0.4758.102 without any issues
ensured that upgrading from 1.37.50 Chromium: 98.0.4758.87 -> 1.37.53 Chromium: 98.0.4758.102 using BraveBrowserNightlySetup32-86-ni-test.exe worked without any issues

Stub installer	Test channel

GeetaSarvadnya · 2022-02-17T18:03:38Z

Covered the DELTA installer verification via https://bravesoftware.slack.com/archives/C0YL5KMA8/p1645118243641749?thread_ts=1645054477.458409&cid=C0YL5KMA8

Nightly

Beta

Release

GeetaSarvadnya · 2022-02-18T08:55:39Z

Enabled logs IsEnabledLogToFile via Regedit key and ensured logs file is showing up the delta installer file brave_installer-delta-x64.exe and upgrade is happening via delta installer, verification is done in the slack thread below:
https://bravesoftware.slack.com/archives/GAA4017R7/p1645173242350309?thread_ts=1645031809.437009&cid=GAA4017R7

simonhong added OS/Windows setup/installer installer/omaha Issues for windows omaha installer labels May 1, 2020

simonhong self-assigned this May 1, 2020

rebron added the OS/Desktop label May 6, 2020

This was referenced May 7, 2020

Support delta brave_installer brave/brave-core#5484

Merged

Add build_delta_installer option to create_dist #9667

Merged

simonhong added the QA/No label May 8, 2020

rebron added the priority/P3 The next thing for us to work on. It'll ride the trains. label May 12, 2020

bsclifton mentioned this issue May 12, 2020

Less expensive upgrade process on macOS (delta installer) #6555

Closed

simonhong added this to the 1.11.x - Nightly milestone May 15, 2020

simonhong closed this as completed in brave/brave-core#5484 May 19, 2020

simonhong mentioned this issue May 23, 2020

Add courgette target to generate courgette.exe in x86 brave/brave-core#5636

Merged

32 tasks

simonhong added the release-notes/exclude label Jun 4, 2020

rebron changed the title ~~Generate delta installer on Windows~~ [Desktop] Support delta installer for updates on Windows Jul 14, 2020

mihaiplesa self-assigned this Jul 16, 2020

mihaiplesa reopened this Jul 16, 2020

bsclifton removed this from the 1.11.x - Release milestone Jul 16, 2020

rebron changed the title ~~[Desktop] Support delta installer for updates on Windows~~ Support delta installer for updates on Windows Oct 8, 2021

mihaiplesa assigned mherrmann Jan 17, 2022

mihaiplesa mentioned this issue Feb 9, 2022

Fix windows delta updates brave/brave-core#12098

Merged

25 tasks

mihaiplesa added this to the 1.37.x - Nightly milestone Feb 9, 2022

mihaiplesa closed this as completed Feb 9, 2022

mihaiplesa added QA/Yes and removed QA/No labels Feb 9, 2022

brave-builds mentioned this issue Feb 10, 2022

Fix windows delta updates (uplift to 1.36.x) brave/brave-core#12210

Merged

7 tasks

mihaiplesa mentioned this issue Feb 10, 2022

Fix windows delta updates (uplift to 1.35.x) brave/brave-core#12211

Merged

7 tasks

kjozwiak modified the milestones: 1.37.x - Nightly, 1.35.x - Release #3 Feb 10, 2022

kjozwiak added QA Pass-Win64 QA Pass-Win86 labels Feb 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support delta installer for updates on Windows #9565

Support delta installer for updates on Windows #9565

simonhong commented May 1, 2020 •

edited

Loading

simonhong commented May 7, 2020 •

edited

Loading

mihaiplesa commented May 11, 2020

simonhong commented May 12, 2020

simonhong commented Jun 4, 2020

mihaiplesa commented Jul 16, 2020

mihaiplesa commented Jan 17, 2022 •

edited

Loading

mihaiplesa commented Feb 9, 2022

GeetaSarvadnya commented Feb 16, 2022 •

edited

Loading

GeetaSarvadnya commented Feb 17, 2022 •

edited

Loading

GeetaSarvadnya commented Feb 18, 2022 •

edited

Loading

Support delta installer for updates on Windows #9565

Support delta installer for updates on Windows #9565

Comments

simonhong commented May 1, 2020 • edited Loading

simonhong commented May 7, 2020 • edited Loading

mihaiplesa commented May 11, 2020

simonhong commented May 12, 2020

simonhong commented Jun 4, 2020

mihaiplesa commented Jul 16, 2020

mihaiplesa commented Jan 17, 2022 • edited Loading

mihaiplesa commented Feb 9, 2022

GeetaSarvadnya commented Feb 16, 2022 • edited Loading

x64 bit stub installers

x32 bit stub installers

GeetaSarvadnya commented Feb 17, 2022 • edited Loading

GeetaSarvadnya commented Feb 18, 2022 • edited Loading

simonhong commented May 1, 2020 •

edited

Loading

simonhong commented May 7, 2020 •

edited

Loading

mihaiplesa commented Jan 17, 2022 •

edited

Loading

GeetaSarvadnya commented Feb 16, 2022 •

edited

Loading

GeetaSarvadnya commented Feb 17, 2022 •

edited

Loading

GeetaSarvadnya commented Feb 18, 2022 •

edited

Loading