Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] feat: Linux support #810

Open
wants to merge 59 commits into
base: linux-support
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
280316c
exclude ios vc by default
ronaldmannak Oct 21, 2022
1850b46
Merge branch 'skywinder:develop' into develop
ronaldmannak Oct 30, 2022
3461cb8
Import FoundationNetworking for Linux
ronaldmannak Oct 30, 2022
773ba92
URLsession to FoundationNetworking
ronaldmannak Oct 30, 2022
f14a899
FoundationNetworking preceeding Foundation
ronaldmannak Oct 30, 2022
33479ee
Add Linux alternative to SecRandomCopyBytes
ronaldmannak Oct 30, 2022
c8fc0a7
Fix return random
ronaldmannak Oct 30, 2022
9898f0d
Fix count
ronaldmannak Oct 30, 2022
5f90f67
Import Vapor
ronaldmannak Oct 30, 2022
8460d92
Add [UInt8].random
ronaldmannak Oct 30, 2022
b7b020d
Fix [UInt8] to Data
ronaldmannak Oct 30, 2022
d4955d4
Excluded WebKit extension for Linux
ronaldmannak Oct 30, 2022
855ad8e
Disabled CoreImage for Linux
ronaldmannak Oct 30, 2022
936ff2a
Comment out random
ronaldmannak Oct 30, 2022
268564f
Rename random to randomData
ronaldmannak Oct 30, 2022
8d7c7df
fix randomData
ronaldmannak Oct 30, 2022
8486802
Add Ubuntu CI workflow
ronaldmannak Oct 30, 2022
44039d2
Rename swift action to ubuntu
ronaldmannak Oct 30, 2022
9e341a3
Add dev/random
ronaldmannak Oct 30, 2022
93a89c1
Use URandom
ronaldmannak Oct 30, 2022
ec86b96
Swift to temp random
ronaldmannak Oct 30, 2022
60b181b
Add try?
ronaldmannak Oct 31, 2022
5976670
Remove force unwraps in unit test
ronaldmannak Oct 31, 2022
84528e2
Force actions to run
ronaldmannak Oct 31, 2022
cdf4ad4
Update Ubuntu actions
ronaldmannak Oct 31, 2022
616168f
Run tests in single step
ronaldmannak Oct 31, 2022
bea91b5
Install ganache
ronaldmannak Oct 31, 2022
ab991d5
Install Node
ronaldmannak Oct 31, 2022
bb0a410
ping localhost
ronaldmannak Nov 1, 2022
71117e3
install nc
ronaldmannak Nov 1, 2022
3cdb528
remote tests only
ronaldmannak Nov 1, 2022
1f51faf
fix yaml error
ronaldmannak Nov 1, 2022
96191e9
Revert to default swift test
ronaldmannak Nov 1, 2022
b7b77a4
fix localhost typo
ronaldmannak Nov 2, 2022
f3c7125
Intentional wrong port. Should fail
ronaldmannak Nov 2, 2022
71ec91d
revert to correct port
ronaldmannak Nov 2, 2022
ce46ea7
Update Ubunti CI script
ronaldmannak Nov 3, 2022
7052ac7
The inevitable “I love YAML” commit
ronaldmannak Nov 3, 2022
9a12130
chore: merged with development
JeneaVranceanu Apr 5, 2023
05a3ee7
Merge branch 'linux-support' into MoonfishApp/develop
JeneaVranceanu Apr 5, 2023
109cceb
feat: BashCommandExecutor implementation added - now we can run opens…
JeneaVranceanu Apr 12, 2023
f767f1c
chore: added tests for ShellCommandExecutor and renamed BashCommandEx…
JeneaVranceanu Apr 12, 2023
7f05f01
chore: removed URandom.swift
JeneaVranceanu Apr 12, 2023
f5d81bc
fix: added new implementation of randomBytes and deprecated old one
JeneaVranceanu Apr 12, 2023
371c603
chore: using new implementation of Data.randomBytes
JeneaVranceanu Apr 12, 2023
264872c
fix: useOpenSSL must be false by default
JeneaVranceanu Apr 12, 2023
4d902fa
chore: ubuntu CI test run on MoonfishApp/develop
JeneaVranceanu Apr 12, 2023
212e246
chore: enabling CI on push to linux-support
JeneaVranceanu Apr 12, 2023
2614fad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 12, 2023
87aabd4
fix: URLSession not available on Linux
JeneaVranceanu Apr 12, 2023
33b805c
Merge branch 'MoonfishApp/develop' of github.com:JeneaVranceanu/web3s…
JeneaVranceanu Apr 12, 2023
b90ddf2
fix: removed static SECP256K1.random function in favour of Data.rando…
JeneaVranceanu Apr 12, 2023
04049ae
fix: tests on Linux - had to import FoundationNetworking
JeneaVranceanu Apr 12, 2023
dcc97cc
fix: tests on Linux - URLResponse init is optional
JeneaVranceanu Apr 12, 2023
ac54678
fix: tests on Linux - URLResponse init requires a set of arguments
JeneaVranceanu Apr 12, 2023
0ec776b
fix: tests on Linux - URL requires an argument
JeneaVranceanu Apr 12, 2023
3764cb9
fix: tests on Linux - URL requires an argument
JeneaVranceanu Apr 12, 2023
d523d5a
fix: URLRequest.init on Linux is not optional
JeneaVranceanu Apr 12, 2023
23f6461
chore: ubuntu CI update - rearranged commands
JeneaVranceanu Apr 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .github/workflows/ubuntu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# This workflow will build a Swift project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-swift

name: Ubuntu

on:
push:
branches:
- master
- develop
- hotfix
- unstable
- linux-support
paths:
- Packag*.swift
- web3swift.podspec
- Cartfile
- Sources/**
- 'Tests/**'
- 'web3swift*/**'
- '.github/workflows/**'
pull_request:
branches:
- master
- develop
- unstable
- MoonfishApp/develop

jobs:
build:

runs-on: ubuntu-latest
container: swift:5.7-focal

steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Resolve dependencies
run: swift package resolve
- name: Build
run: swift build
- name: Install ganache
run: npm install ganache --global
- name: install nc
run: apt-get update && apt-get install -y netcat
- name: Start ganache in background
run: ganache &
- name: Wait till ganache starts
run: sleep 1
- name: Ping
run: nc -vz 127.0.0.1 8545
- name: Run tests
run: swift test
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

import Foundation
import BigInt
#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

extension APIRequest {
public static func sendRequest<Result>(with provider: Web3Provider, for call: APIRequest) async throws -> APIResponse<Result> {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
//

import Foundation
#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

@available(iOS, obsoleted: 15.0, message: "Use the built-in API instead")
@available(macOS, obsoleted: 12.0, message: "Use the built-in API instead")
Expand Down
8 changes: 2 additions & 6 deletions Sources/Web3Core/KeystoreManager/BIP32Keystore.swift
Original file line number Diff line number Diff line change
Expand Up @@ -220,17 +220,13 @@ public class BIP32Keystore: AbstractKeystore {
guard data.count == 82 else {
throw AbstractKeystoreError.encryptionError("Invalid expected data length")
}
guard let saltData = Data.randomBytes(length: 32) else {
throw AbstractKeystoreError.noEntropyError
}
let saltData = try Data.randomBytes(count: 32)
guard let derivedKey = scrypt(password: password, salt: saltData, length: dkLen, N: N, R: R, P: P) else {
throw AbstractKeystoreError.keyDerivationError
}
let last16bytes = derivedKey[(derivedKey.count - 16)...(derivedKey.count - 1)]
let encryptionKey = derivedKey[0...15]
guard let IV = Data.randomBytes(length: 16) else {
throw AbstractKeystoreError.noEntropyError
}
let IV = try Data.randomBytes(count: 16)
var aesCipher: AES?
switch aesMode {
case "aes-128-cbc":
Expand Down
7 changes: 2 additions & 5 deletions Sources/Web3Core/KeystoreManager/BIP39.swift
Original file line number Diff line number Diff line change
Expand Up @@ -95,13 +95,10 @@ public class BIP39 {
}

private static func entropyOf(size: Int) throws -> Data {
guard
size >= 128 && size <= 256 && size.isMultiple(of: 32),
let entropy = Data.randomBytes(length: size/8)
else {
guard size >= 128 && size <= 256 && size.isMultiple(of: 32) else {
throw AbstractKeystoreError.noEntropyError
}
return entropy
return try Data.randomBytes(count: size / 8)
}

static func bitarray(from data: Data) -> String {
Expand Down
8 changes: 2 additions & 6 deletions Sources/Web3Core/KeystoreManager/EthereumKeystoreV3.swift
Original file line number Diff line number Diff line change
Expand Up @@ -95,17 +95,13 @@ public class EthereumKeystoreV3: AbstractKeystore {
throw AbstractKeystoreError.encryptionError("Encryption without key data")
}
let saltLen = 32
guard let saltData = Data.randomBytes(length: saltLen) else {
throw AbstractKeystoreError.noEntropyError
}
let saltData = try Data.randomBytes(count: saltLen)
guard let derivedKey = scrypt(password: password, salt: saltData, length: dkLen, N: N, R: R, P: P) else {
throw AbstractKeystoreError.keyDerivationError
}
let last16bytes = Data(derivedKey[(derivedKey.count - 16)...(derivedKey.count - 1)])
let encryptionKey = Data(derivedKey[0...15])
guard let IV = Data.randomBytes(length: 16) else {
throw AbstractKeystoreError.noEntropyError
}
let IV = try Data.randomBytes(count: 16)
var aesCipher: AES?
switch aesMode {
case "aes-128-cbc":
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
//

import Foundation
#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

public struct EtherscanTransactionChecker: TransactionChecker {
private let urlSession: URLSessionProxy
Expand Down
24 changes: 2 additions & 22 deletions Sources/Web3Core/Structure/SECP256k1.swift
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ extension SECP256K1 {
return nil
}
var recoverableSignature: secp256k1_ecdsa_recoverable_signature = secp256k1_ecdsa_recoverable_signature()
guard let extraEntropy = SECP256K1.randomBytes(length: 32) else { return nil }
guard let extraEntropy = try? Data.randomBytes(count: 32) else { return nil }
let result = hash.withUnsafeBytes { hashRBPointer -> Int32? in
if let hashRPointer = hashRBPointer.baseAddress, hashRBPointer.count > 0 {
let hashPointer = hashRPointer.assumingMemoryBound(to: UInt8.self)
Expand Down Expand Up @@ -303,7 +303,7 @@ extension SECP256K1 {

public static func generatePrivateKey() -> Data? {
for _ in 0...1024 {
guard let keyData = SECP256K1.randomBytes(length: 32) else {
guard let keyData = try? Data.randomBytes(count: 32) else {
continue
}
guard SECP256K1.verifyPrivateKey(privateKey: keyData) else {
Expand Down Expand Up @@ -338,26 +338,6 @@ extension SECP256K1 {
return completeSignature
}

internal static func randomBytes(length: Int) -> Data? {
for _ in 0...1024 {
var data = Data(repeating: 0, count: length)
let result = data.withUnsafeMutableBytes { mutableRBBytes -> Int32? in
if let mutableRBytes = mutableRBBytes.baseAddress, mutableRBBytes.count > 0 {
let mutableBytes = mutableRBytes.assumingMemoryBound(to: UInt8.self)
return SecRandomCopyBytes(kSecRandomDefault, length, mutableBytes)
} else {
return nil
}
}
if let res = result, res == errSecSuccess {
return data
} else {
continue
}
}
return nil
}

internal static func toByteArray<T>(_ value: T) -> [UInt8] {
var value = value
return withUnsafeBytes(of: &value) { Array($0) }
Expand Down
3 changes: 3 additions & 0 deletions Sources/Web3Core/Structure/Web3ProviderProtocol.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
//

import Foundation
#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

public protocol Web3Provider {
var network: Networks? {get set}
Expand Down
52 changes: 41 additions & 11 deletions Sources/Web3Core/Utility/Data+Extension.swift
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import Foundation


extension Data {
init<T>(fromArray values: [T]) {
let values = values
Expand Down Expand Up @@ -40,22 +41,51 @@ extension Data {
}
}

@available(*, deprecated, message: "Please, use throwing `randomBytes(count)` function instead to get information instead of `nil` value on why the function call failed.")
/// Runs `SecRandomCopyBytes` for Apple platforms and `openssl rand -hex` for other platforms
/// to generate cryptographically secure random bytes.
/// - Parameter count: how many bytes to generate. Value below or equal to 0 will return `nil`.
/// - Returns: random bytes or `nil`.
public static func randomBytes(length: Int) -> Data? {
for _ in 0...1024 {
var data = Data(repeating: 0, count: length)
let result = data.withUnsafeMutableBytes { (body: UnsafeMutableRawBufferPointer) -> Int32? in
if let bodyAddress = body.baseAddress, body.count > 0 {
try? randomBytes(count: length)
}

/// Runs `SecRandomCopyBytes` for Apple platforms and `openssl rand -hex` for other platforms
/// to generate cryptographically secure random bytes.
/// - Parameter count: how many bytes to generate. Value below or equal to 0 will throw an error.
/// - Parameter useOpenSSL: **has no effect on Linux and Windows**. When set to `true` forces the use of external executable `openssl`. It's your responsibility to make sure openssl is installed on this machine. By default set to `false`. To install follow [the official guide](https://www.openssl.org/source/ ).
/// - Returns: random bytes or throws an error,
public static func randomBytes(count: Int, useOpenSSL: Bool = false) throws -> Data {
guard count > 0 else { throw Web3Error.valueError(desc: "Cannot generate \(count) random bytes.") }

#if !(os(Linux) || os(Windows))
if !useOpenSSL {
for _ in 0...1024 {
var data = Data(repeating: 0, count: count)
let result = try data.withUnsafeMutableBytes { (body: UnsafeMutableRawBufferPointer) -> Int32? in
guard let bodyAddress = body.baseAddress, body.count == count else {
throw Web3Error.processingError(desc: "Address of the buffer is nil (\(body.baseAddress == nil)) or the count of bytes in the buffer is not equal to the count requested by the user (\(body.count < count)).")
}

let pointer = bodyAddress.assumingMemoryBound(to: UInt8.self)
return SecRandomCopyBytes(kSecRandomDefault, length, pointer)
} else {
return nil
return SecRandomCopyBytes(kSecRandomDefault, count, pointer)
}
if let notNilResult = result, notNilResult == errSecSuccess {
return data
}
}
if let notNilResult = result, notNilResult == errSecSuccess {
return data
}
}
return nil
#endif
let randomBytesHex = try ShellCommandExecutor().run(commandName: "openssl rand -hex \(count)")
guard let bytes = Data.fromHex(randomBytesHex) else {
throw Web3Error.processingError(desc: "Random bytes generated by the openssl are in an invalid hex representation: \(randomBytesHex)")
}

guard bytes.count == count else {
throw Web3Error.processingError(desc: "Count of generated by the openssl random bytes is not equal to the count requested by the user: expected \(count), given \(bytes.count).")
}

return bytes
}

public func bitsInRange(_ startingBit: Int, _ length: Int) -> UInt64? { // return max of 8 bytes for simplicity, non-public
Expand Down
67 changes: 67 additions & 0 deletions Sources/Web3Core/Utility/ShellCommandExecutor.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
//
// ShellCommandExecutor.swift
//
// Created by JeneaVranceanu on 05.04.2023.
//

import Foundation

internal typealias ShellCommandExecutor = BashCommandExecutor

internal enum ShellError: Error {
case commandNotFound(name: String)
}

internal struct BashCommandExecutor {
func run(commandName: String, arguments: [String] = []) throws -> String {
var arguments = arguments
var command = commandName
if commandName.contains(" ") {
var args = commandName.trim().split(separator: " ")
command = String(args.removeFirst())
arguments.append(contentsOf: args.map { String($0) })
}
return try run(resolve(command), with: arguments)
}

func run(commandName: String, arguments: [String] = []) async throws -> String {
try await withCheckedThrowingContinuation { continuation in
do {
continuation.resume(returning: try run(commandName: commandName, arguments: arguments))
} catch {
continuation.resume(throwing: error)
}
}
}


func resolve(_ command: String) throws -> String {
#if os(Windows)
// TODO: add a check to make sure command exists on Windows
return command
#else
let shellCommand = try run("/bin/bash",
with: ["-l", "-c", "which \(command)"])
.trimmingCharacters(in: .whitespacesAndNewlines)

guard !shellCommand.isEmpty else {
throw ShellError.commandNotFound(name: command)
}
return shellCommand
#endif
}

func run(_ command: String, with arguments: [String] = []) throws -> String {
let process = Process()
process.launchPath = command
process.arguments = arguments
let outputPipe = Pipe()
process.standardOutput = outputPipe
try process.run()
let outputData = outputPipe.fileHandleForReading.readDataToEndOfFile()
guard let output = String(bytes: outputData, encoding: .utf8)?.trim() else {
throw Web3Error.valueError(desc: "Shell command returned bytes that cannot be decoded as UTF-8: \(outputData.toHexString())")
}
return output
}
}
2 changes: 2 additions & 0 deletions Sources/web3swift/Browser/Bridge.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
// Copyright © 2017 Samaritan. All rights reserved.
//

#if !os(Linux)
import WebKit

/// Bridge for WKWebView and JavaScript
Expand Down Expand Up @@ -245,3 +246,4 @@ fileprivate extension WKWebView {
evaluateJavaScript(jsString, completionHandler: completionHandler)
}
}
#endif
6 changes: 6 additions & 0 deletions Sources/web3swift/Utils/EIP/EIP67Code.swift
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
//

import Foundation
#if !os(Linux)
import CoreImage
#endif
import BigInt
import Web3Core

Expand Down Expand Up @@ -77,11 +79,14 @@ extension Web3 {
return mainPart
}

#if !os(Linux)
public func toImage(scale: Double = 1.0) -> CIImage {
return EIP67CodeGenerator.createImage(from: self, scale: scale)
}
#endif
}

#if !os(Linux)
public struct EIP67CodeGenerator {

public static func createImage(from: EIP67Code, scale: Double = 1.0) -> CIImage {
Expand All @@ -94,6 +99,7 @@ extension Web3 {
return image
}
}
#endif

public struct EIP67CodeParser {
public static func parse(_ data: Data) -> EIP67Code? {
Expand Down
3 changes: 3 additions & 0 deletions Sources/web3swift/Web3/Web3+HttpProvider.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import Foundation
import BigInt
import Web3Core
#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

/// The default http provider.
public class Web3HttpProvider: Web3Provider {
Expand Down
Loading