Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup log list v1 dependencies #977

Merged
merged 11 commits into from
Sep 10, 2022
Merged

Cleanup log list v1 dependencies #977

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
5847e78
Remove ctutil/loginfo.go
roger2hk Sep 8, 2022
ec3a97a
Migrate loglist from v1 to v3 in ctclient cmd
roger2hk Sep 8, 2022
f2701df
Update CHANGELOG.md
roger2hk Sep 8, 2022
ff3a5b5
Update CHANGELOG.md
roger2hk Sep 8, 2022
b65b71b
Remove log list v1 package and its dependencies
roger2hk Sep 8, 2022
471a434
Migrate loglist dependency from v1 to v3 in ctutil/loginfo.go
roger2hk Sep 8, 2022
0c44577
Update CHANGELOG.md
roger2hk Sep 8, 2022
e112e1b
Migrate loglist dependency from v1 to v3 in ctutil/sctscan.go
roger2hk Sep 8, 2022
69e3913
Migrate loglist dependency from v1 to v3 in trillian/integration/ct_h…
roger2hk Sep 8, 2022
a8615ba
Update CHANGELOG.md
roger2hk Sep 8, 2022
9ba1f44
go mod tidy
roger2hk Sep 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@

### Cleanup

* `WithBalancerName` is deprecated and removed, using the recommended way
* `WithBalancerName` is deprecated and removed, using the recommended way.
* `ctfe.PEMCertPool` type has been moved to `x509util.PEMCertPool` to reduce
dependencies (#903).
* Remove log list v1 package and its dependencies.

### Migrillian

Expand All @@ -29,6 +30,10 @@
* #800: Remove dependency from `ratelimit`.
* #927: Add read-only mode to CTFE config.
* Update Trillian to [0a389c4](https://github.com/google/trillian/commit/0a389c4bb8d97fb3be8f55d7e5b428cf4304986f)
* Migrate loglist dependency from v1 to v3 in ctclient cmd.
* Migrate loglist dependency from v1 to v3 in ctutil/loginfo.go
* Migrate loglist dependency from v1 to v3 in ctutil/sctscan.go
* Migrate loglist dependency from v1 to v3 in trillian/integration/ct_hammer/main.go

## v1.1.2

Expand Down
6 changes: 3 additions & 3 deletions client/ctclient/cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ import (
ct "github.com/google/certificate-transparency-go"
"github.com/google/certificate-transparency-go/client"
"github.com/google/certificate-transparency-go/jsonclient"
"github.com/google/certificate-transparency-go/loglist"
"github.com/google/certificate-transparency-go/loglist3"
"github.com/google/certificate-transparency-go/x509util"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
Expand All @@ -53,7 +53,7 @@ func init() {
flags := rootCmd.PersistentFlags()
flags.BoolVar(&skipHTTPSVerify, "skip_https_verify", false, "Skip verification of HTTPS transport connection")
flags.StringVar(&logName, "log_name", "", "Name of log to retrieve information from --log_list for")
flags.StringVar(&logList, "log_list", loglist.AllLogListURL, "Location of master log list (URL or filename)")
flags.StringVar(&logList, "log_list", loglist3.AllLogListURL, "Location of master log list (URL or filename)")
flags.StringVar(&logURI, "log_uri", "https://ct.googleapis.com/rocketeer", "CT log base URI")
flags.StringVar(&pubKey, "pub_key", "", "Name of file containing log's public key")
}
Expand Down Expand Up @@ -121,7 +121,7 @@ func connect(ctx context.Context) *client.LogClient {
if err != nil {
klog.Exitf("Failed to read log list: %v", err)
}
ll, err := loglist.NewFromJSON(llData)
ll, err := loglist3.NewFromJSON(llData)
if err != nil {
klog.Exitf("Failed to build log list: %v", err)
}
Expand Down
26 changes: 14 additions & 12 deletions ctutil/loginfo.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import (
ct "github.com/google/certificate-transparency-go"
"github.com/google/certificate-transparency-go/client"
"github.com/google/certificate-transparency-go/jsonclient"
"github.com/google/certificate-transparency-go/loglist"
"github.com/google/certificate-transparency-go/loglist3"
"github.com/google/certificate-transparency-go/x509"
"github.com/transparency-dev/merkle/proof"
"github.com/transparency-dev/merkle/rfc6962"
Expand All @@ -46,7 +46,7 @@ type LogInfo struct {
}

// NewLogInfo builds a LogInfo object based on a log list entry.
func NewLogInfo(log *loglist.Log, hc *http.Client) (*LogInfo, error) {
func NewLogInfo(log *loglist3.Log, hc *http.Client) (*LogInfo, error) {
url := log.URL
if !strings.HasPrefix(url, "https://") {
url = "https://" + url
Expand All @@ -58,7 +58,7 @@ func NewLogInfo(log *loglist.Log, hc *http.Client) (*LogInfo, error) {
return newLogInfo(log, lc)
}

func newLogInfo(log *loglist.Log, lc client.CheckLogClient) (*LogInfo, error) {
func newLogInfo(log *loglist3.Log, lc client.CheckLogClient) (*LogInfo, error) {
logKey, err := x509.ParsePKIXPublicKey(log.Key)
if err != nil {
return nil, fmt.Errorf("failed to parse public key data for log %q: %v", log.Description, err)
Expand All @@ -67,7 +67,7 @@ func newLogInfo(log *loglist.Log, lc client.CheckLogClient) (*LogInfo, error) {
if err != nil {
return nil, fmt.Errorf("failed to build verifier log %q: %v", log.Description, err)
}
mmd := time.Duration(log.MaximumMergeDelay) * time.Second
mmd := time.Duration(log.MMD) * time.Second
return &LogInfo{
Description: log.Description,
Client: lc,
Expand All @@ -81,19 +81,21 @@ func newLogInfo(log *loglist.Log, lc client.CheckLogClient) (*LogInfo, error) {
type LogInfoByHash map[[sha256.Size]byte]*LogInfo

// LogInfoByKeyHash builds a map of LogInfo objects indexed by their key hashes.
func LogInfoByKeyHash(ll *loglist.LogList, hc *http.Client) (LogInfoByHash, error) {
func LogInfoByKeyHash(ll *loglist3.LogList, hc *http.Client) (LogInfoByHash, error) {
return logInfoByKeyHash(ll, hc, NewLogInfo)
}

func logInfoByKeyHash(ll *loglist.LogList, hc *http.Client, infoFactory func(*loglist.Log, *http.Client) (*LogInfo, error)) (map[[sha256.Size]byte]*LogInfo, error) {
func logInfoByKeyHash(ll *loglist3.LogList, hc *http.Client, infoFactory func(*loglist3.Log, *http.Client) (*LogInfo, error)) (map[[sha256.Size]byte]*LogInfo, error) {
result := make(map[[sha256.Size]byte]*LogInfo)
for _, log := range ll.Logs {
h := sha256.Sum256(log.Key)
li, err := infoFactory(&log, hc)
if err != nil {
return nil, err
for _, operator := range ll.Operators {
for _, log := range operator.Logs {
h := sha256.Sum256(log.Key)
li, err := infoFactory(log, hc)
if err != nil {
return nil, err
}
result[h] = li
}
result[h] = li
}
return result, nil
}
Expand Down
16 changes: 8 additions & 8 deletions ctutil/sctcheck/sctcheck.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import (
"time"

"github.com/google/certificate-transparency-go/ctutil"
"github.com/google/certificate-transparency-go/loglist"
"github.com/google/certificate-transparency-go/loglist3"
"github.com/google/certificate-transparency-go/x509"
"github.com/google/certificate-transparency-go/x509util"
"k8s.io/klog/v2"
Expand All @@ -39,12 +39,12 @@ import (
)

var (
logList = flag.String("log_list", loglist.AllLogListURL, "Location of master CT log list (URL or filename)")
logList = flag.String("log_list", loglist3.AllLogListURL, "Location of master CT log list (URL or filename)")
deadline = flag.Duration("deadline", 30*time.Second, "Timeout deadline for HTTP requests")
checkInclusion = flag.Bool("check_inclusion", true, "Whether to check SCT inclusion in issuing CT log")
)

type logInfoFactory func(*loglist.Log, *http.Client) (*ctutil.LogInfo, error)
type logInfoFactory func(*loglist3.Log, *http.Client) (*ctutil.LogInfo, error)

func main() {
klog.InitFlags(nil)
Expand All @@ -56,7 +56,7 @@ func main() {
if err != nil {
klog.Exitf("Failed to read log list: %v", err)
}
ll, err := loglist.NewFromJSON(llData)
ll, err := loglist3.NewFromJSON(llData)
if err != nil {
klog.Exitf("Failed to parse log list: %v", err)
}
Expand Down Expand Up @@ -106,7 +106,7 @@ func main() {

// checkChain iterates over any embedded SCTs in the leaf certificate of the chain
// and checks those SCTs. Returns the counts of valid and invalid embedded SCTs found.
func checkChain(ctx context.Context, lf logInfoFactory, chain []*x509.Certificate, ll *loglist.LogList, hc *http.Client) (int, int) {
func checkChain(ctx context.Context, lf logInfoFactory, chain []*x509.Certificate, ll *loglist3.LogList, hc *http.Client) (int, int) {
leaf := chain[0]
if len(leaf.SCTList.SCTList) == 0 {
return 0, 0
Expand Down Expand Up @@ -148,7 +148,7 @@ func checkChain(ctx context.Context, lf logInfoFactory, chain []*x509.Certificat
// for an HTTPS site. Along the way it checks any external SCTs that are served
// up on the connection alongside the chain. Returns the chain and counts of
// valid and invalid external SCTs found.
func getAndCheckSiteChain(ctx context.Context, lf logInfoFactory, target string, ll *loglist.LogList, hc *http.Client) ([]*x509.Certificate, int, int, error) {
func getAndCheckSiteChain(ctx context.Context, lf logInfoFactory, target string, ll *loglist3.LogList, hc *http.Client) ([]*x509.Certificate, int, int, error) {
u, err := url.Parse(target)
if err != nil {
return nil, 0, 0, fmt.Errorf("failed to parse URL: %v", err)
Expand Down Expand Up @@ -212,7 +212,7 @@ func getAndCheckSiteChain(ctx context.Context, lf logInfoFactory, target string,
// checkSCT performs checks on an SCT and Merkle tree leaf, performing both
// signature validation and online log inclusion checking. Returns whether
// the SCT is valid.
func checkSCT(ctx context.Context, liFactory logInfoFactory, subject string, merkleLeaf *ct.MerkleTreeLeaf, sctData *x509.SerializedSCT, ll *loglist.LogList, hc *http.Client) bool {
func checkSCT(ctx context.Context, liFactory logInfoFactory, subject string, merkleLeaf *ct.MerkleTreeLeaf, sctData *x509.SerializedSCT, ll *loglist3.LogList, hc *http.Client) bool {
sct, err := x509util.ExtractSCT(sctData)
if err != nil {
klog.Errorf("Failed to deserialize %s data: %v", subject, err)
Expand Down Expand Up @@ -246,7 +246,7 @@ func checkSCT(ctx context.Context, liFactory logInfoFactory, subject string, mer
if err != nil {
age := time.Since(ct.TimestampToTime(sct.Timestamp))
if age < logInfo.MMD {
klog.Warningf("Failed to verify inclusion proof (%v) but %s timestamp is only %v old, less than log's MMD of %d seconds", err, subject, age, log.MaximumMergeDelay)
klog.Warningf("Failed to verify inclusion proof (%v) but %s timestamp is only %v old, less than log's MMD of %d seconds", err, subject, age, log.MMD)
} else {
klog.Errorf("Failed to verify inclusion proof for %s: %v", subject, err)
}
Expand Down
6 changes: 3 additions & 3 deletions ctutil/sctscan/sctscan.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import (
"github.com/google/certificate-transparency-go/client"
"github.com/google/certificate-transparency-go/ctutil"
"github.com/google/certificate-transparency-go/jsonclient"
"github.com/google/certificate-transparency-go/loglist"
"github.com/google/certificate-transparency-go/loglist3"
"github.com/google/certificate-transparency-go/scanner"
"github.com/google/certificate-transparency-go/x509"
"github.com/google/certificate-transparency-go/x509util"
Expand All @@ -36,7 +36,7 @@ import (

var (
logURI = flag.String("log_uri", "https://ct.googleapis.com/pilot", "CT log base URI")
logList = flag.String("log_list", loglist.AllLogListURL, "Location of master CT log list (URL or filename)")
logList = flag.String("log_list", loglist3.AllLogListURL, "Location of master CT log list (URL or filename)")
inclusion = flag.Bool("inclusion", false, "Whether to do inclusion checking")
deadline = flag.Duration("deadline", 30*time.Second, "Timeout deadline for HTTP requests")
batchSize = flag.Int("batch_size", 1000, "Max number of entries to request at per call to get-entries")
Expand Down Expand Up @@ -71,7 +71,7 @@ func main() {
if err != nil {
klog.Exitf("Failed to read log list: %v", err)
}
ll, err := loglist.NewFromJSON(llData)
ll, err := loglist3.NewFromJSON(llData)
if err != nil {
klog.Exitf("Failed to parse log list: %v", err)
}
Expand Down
1 change: 0 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ require (
github.com/gorilla/mux v1.8.0
github.com/kylelemons/godebug v1.1.0
github.com/mattn/go-sqlite3 v1.14.15
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826
github.com/prometheus/client_golang v1.13.0
github.com/rs/cors v1.8.2
github.com/sergi/go-diff v1.2.0
Expand Down
2 changes: 0 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,8 +452,6 @@ github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lN
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 h1:RWengNIwukTxcDr9M+97sNutRR1RKhG96O6jWumTTnw=
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-proto-validators v0.0.0-20180403085117-0950a7990007/go.mod h1:m2XC9Qq0AlmmVksL6FktJCdTYyLk7V3fKyp0sl1yWQo=
Expand Down
14 changes: 0 additions & 14 deletions loglist/chrome-list-pubkey.pem
View file
Open in desktop

This file was deleted.

114 changes: 0 additions & 114 deletions loglist/diff_check.go
View file
Open in desktop

This file was deleted.

Loading