Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to be committed: #223

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

# Package Analysis

The Package Analysis project analyses the capabilities of packages available on open source repositories. The project looks for behaviors that indicate malicious software:
The Package Analysis project analyses the capabilities of packages available on open source repositories. The project looks for behaviors that indicate malicious software:

- What files do they access?
- What addresses do they connect to?
- What commands do they run?
- What files do they access?
- What addresses do they connect to?
- What commands do they run?

The project also tracks changes in how packages behave over time, to identify when previously safe software begins acting suspiciously.
The project also tracks changes in how packages behave over time, to identify when previously safe software begins acting suspiciously.

This effort is meant to improve the security of open source software by detecting malicious behavior, informing consumers selecting packages, and providing researchers with data about the ecosystem.
This effort is meant to improve the security of open source software by detecting malicious behavior, informing consumers selecting packages, and providing researchers with data about the ecosystem.

This code is designed to work with the
[Package Feeds](https://github.com/khulnasoft-lab/package-feeds) project,
Expand Down Expand Up @@ -52,6 +52,7 @@ as network connections that can be used to leak sensitive data or allow remote
access.

## Public Data

This data is available in the public [BigQuery dataset](https://console.cloud.google.com/bigquery?d=packages&p=khulnasoft-malware-analysis&t=analysis&page=table).

## Configuration
Expand All @@ -75,15 +76,15 @@ An example of these variables can be found in the
the data coming out of scheduler. Values should follow
[goclouddev subscriptions](https://gocloud.dev/howto/pubsub/subscribe/).

`OSSF_MALWARE_ANALYSIS_RESULTS` - **OPTIONAL**: Can be used to set the bucket
`KHULNASOFT_MALWARE_ANALYSIS_RESULTS` - **OPTIONAL**: Can be used to set the bucket
URL to publish results to. Values should follow
[goclouddev buckets](https://gocloud.dev/howto/blob/).

`OSSF_MALWARE_ANALYSIS_PACKAGES` - **OPTIONAL**: Can be used to set the bucket
`KHULNASOFT_MALWARE_ANALYSIS_PACKAGES` - **OPTIONAL**: Can be used to set the bucket
URL to get custom uploaded packages from. Values should follow
[goclouddev buckets](https://gocloud.dev/howto/blob/).

`OSSF_MALWARE_NOTIFICATION_TOPIC` - **OPTIONAL**: Can be used to set the topic URL to
`KHULNASOFT_MALWARE_NOTIFICATION_TOPIC` - **OPTIONAL**: Can be used to set the topic URL to
publish messages for consumption after a new package analysis is complete. Values should follow
[goclouddev publishing](https://gocloud.dev/howto/pubsub/publish/).

Expand Down Expand Up @@ -132,7 +133,6 @@ $ scripts/run_analysis.sh -ecosystem pypi -package Django -version 4.1.3
To run analysis on a local PyPi package named 'test',
located in local archive `/path/to/test.whl`


```bash
$ scripts/run_analysis.sh -ecosystem pypi -package test -local /path/to/test.whl
```
Expand All @@ -148,6 +148,7 @@ allows caching the sandbox images and supports local developement.
## Development

### Testing

See `sample_packages/README.md` for how to use a sample package that simulates malicious activity for testing purposes.

### Required Dependencies
Expand All @@ -157,6 +158,6 @@ See `sample_packages/README.md` for how to use a sample package that simulates m

# Contributing

If you want to get involved or have ideas you'd like to chat about, we discuss this project in the [OSSF Securing Critical Projects Working Group](https://github.com/ossf/wg-securing-critical-projects) meetings.
If you want to get involved or have ideas you'd like to chat about, we discuss this project in the [KHULNASOFT Securing Critical Projects Working Group](https://github.com/ossf/wg-securing-critical-projects) meetings.

See the [Community Calendar](https://calendar.google.com/calendar?cid=czYzdm9lZmhwNWk5cGZsdGI1cTY3bmdwZXNAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ) for the schedule and meeting invitations.
2 changes: 1 addition & 1 deletion cmd/analyze/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@ RUN chmod 755 /usr/local/bin/runsc_compat.sh && \
chmod 644 /usr/local/etc/iptables.rules /etc/cni/net.d/podman-analysis.conflist

ARG SANDBOX_IMAGE_TAG
ENV OSSF_SANDBOX_IMAGE_TAG=${SANDBOX_IMAGE_TAG}
ENV KHULNASOFT_SANDBOX_IMAGE_TAG=${SANDBOX_IMAGE_TAG}
20 changes: 10 additions & 10 deletions cmd/worker/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,20 +61,20 @@ func resultStoreForEnv(key string) *resultstore.ResultStore {
func configFromEnv() *config {
return &config{
imageSpec: sandboxImageSpec{
tag: os.Getenv("OSSF_SANDBOX_IMAGE_TAG"),
noPull: os.Getenv("OSSF_SANDBOX_NOPULL") != "",
tag: os.Getenv("KHULNASOFT_SANDBOX_IMAGE_TAG"),
noPull: os.Getenv("KHULNASOFT_SANDBOX_NOPULL") != "",
},
resultStores: &worker.ResultStores{
AnalyzedPackage: resultStoreForEnv("OSSF_MALWARE_ANALYZED_PACKAGES"),
DynamicAnalysis: resultStoreForEnv("OSSF_MALWARE_ANALYSIS_RESULTS"),
ExecutionLog: resultStoreForEnv("OSSF_MALWARE_ANALYSIS_EXECUTION_LOGS"),
FileWrites: resultStoreForEnv("OSSF_MALWARE_ANALYSIS_FILE_WRITE_RESULTS"),
StaticAnalysis: resultStoreForEnv("OSSF_MALWARE_STATIC_ANALYSIS_RESULTS"),
AnalyzedPackage: resultStoreForEnv("KHULNASOFT_MALWARE_ANALYZED_PACKAGES"),
DynamicAnalysis: resultStoreForEnv("KHULNASOFT_MALWARE_ANALYSIS_RESULTS"),
ExecutionLog: resultStoreForEnv("KHULNASOFT_MALWARE_ANALYSIS_EXECUTION_LOGS"),
FileWrites: resultStoreForEnv("KHULNASOFT_MALWARE_ANALYSIS_FILE_WRITE_RESULTS"),
StaticAnalysis: resultStoreForEnv("KHULNASOFT_MALWARE_STATIC_ANALYSIS_RESULTS"),
},
subURL: os.Getenv("OSSMALWARE_WORKER_SUBSCRIPTION"),
packagesBucket: os.Getenv("OSSF_MALWARE_ANALYSIS_PACKAGES"),
notificationTopicURL: os.Getenv("OSSF_MALWARE_NOTIFICATION_TOPIC"),
packagesBucket: os.Getenv("KHULNASOFT_MALWARE_ANALYSIS_PACKAGES"),
notificationTopicURL: os.Getenv("KHULNASOFT_MALWARE_NOTIFICATION_TOPIC"),

userAgentExtra: os.Getenv("OSSF_MALWARE_USER_AGENT_EXTRA"),
userAgentExtra: os.Getenv("KHULNASOFT_MALWARE_USER_AGENT_EXTRA"),
}
}
6 changes: 3 additions & 3 deletions cmd/worker/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ func handleMessage(ctx context.Context, msg *pubsub.Message, cfg *config, packag

// propogate user agent extras to the static analysis sandbox if it is set.
if cfg.userAgentExtra != "" {
staticSandboxOpts = append(staticSandboxOpts, sandbox.SetEnv("OSSF_MALWARE_USER_AGENT_EXTRA", cfg.userAgentExtra))
staticSandboxOpts = append(staticSandboxOpts, sandbox.SetEnv("KHULNASOFT_MALWARE_USER_AGENT_EXTRA", cfg.userAgentExtra))
}

// run both dynamic and static analysis regardless of error status of either
Expand Down Expand Up @@ -241,7 +241,7 @@ func main() {

http.DefaultTransport = useragent.DefaultRoundTripper(http.DefaultTransport, cfg.userAgentExtra)

if err := featureflags.Update(os.Getenv("OSSF_MALWARE_FEATURE_FLAGS")); err != nil {
if err := featureflags.Update(os.Getenv("KHULNASOFT_MALWARE_FEATURE_FLAGS")); err != nil {
slog.Error("Failed to parse feature flags", "error", err)
os.Exit(1)
}
Expand All @@ -250,7 +250,7 @@ func main() {

// If configured, start a webserver so that Go's pprof can be accessed for
// debugging and profiling.
if os.Getenv("OSSF_MALWARE_ANALYSIS_ENABLE_PROFILER") != "" {
if os.Getenv("KHULNASOFT_MALWARE_ANALYSIS_ENABLE_PROFILER") != "" {
go func() {
slog.Info("Starting profiler")
http.ListenAndServe(":6060", nil)
Expand Down
16 changes: 8 additions & 8 deletions configs/e2e/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,14 @@ services:
entrypoint: "/usr/local/bin/worker"
environment:
OSSMALWARE_WORKER_SUBSCRIPTION: kafka://worker?topic=workers
OSSF_MALWARE_NOTIFICATION_TOPIC: kafka://notifications
OSSF_MALWARE_ANALYZED_PACKAGES: s3://package-analysis/analyzed-packages?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
OSSF_MALWARE_ANALYSIS_RESULTS: s3://package-analysis/dynamic?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
OSSF_MALWARE_ANALYSIS_EXECUTION_LOGS: s3://package-analysis/execution-logs?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
OSSF_MALWARE_ANALYSIS_FILE_WRITE_RESULTS: s3://package-analysis/file-writes?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
OSSF_MALWARE_STATIC_ANALYSIS_RESULTS: s3://package-analysis/static?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
OSSF_MALWARE_ANALYSIS_ENABLE_PROFILER: "true"
OSSF_MALWARE_FEATURE_FLAGS: ""
KHULNASOFT_MALWARE_NOTIFICATION_TOPIC: kafka://notifications
KHULNASOFT_MALWARE_ANALYZED_PACKAGES: s3://package-analysis/analyzed-packages?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
KHULNASOFT_MALWARE_ANALYSIS_RESULTS: s3://package-analysis/dynamic?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
KHULNASOFT_MALWARE_ANALYSIS_EXECUTION_LOGS: s3://package-analysis/execution-logs?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
KHULNASOFT_MALWARE_ANALYSIS_FILE_WRITE_RESULTS: s3://package-analysis/file-writes?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
KHULNASOFT_MALWARE_STATIC_ANALYSIS_RESULTS: s3://package-analysis/static?endpoint=minio:9000&disableSSL=true&s3ForcePathStyle=true
KHULNASOFT_MALWARE_ANALYSIS_ENABLE_PROFILER: "true"
KHULNASOFT_MALWARE_FEATURE_FLAGS: ""
KAFKA_BROKERS: kafka:9092
AWS_ACCESS_KEY_ID: minio
AWS_SECRET_ACCESS_KEY: minio123
Expand Down
18 changes: 9 additions & 9 deletions docs/case_studies.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Discord attacks are focused on attacking [Discord](https://discord.com) accounts

### PyPI: discordcmd

*2022-02-18*, [Analysis Result](https://storage.googleapis.com/ossf-malware-analysis-results/pypi/discordcmd/0.0.2.json)
_2022-02-18_, [Analysis Result](https://storage.googleapis.com/khulnasoft-malware-analysis-results/pypi/discordcmd/0.0.2.json)

This Python package will attack the desktop client for Discord on Windows. It was found by spotting the unusual requests to raw.githubusercontent.com, Discord API, and ipinfo.io.

Expand All @@ -20,11 +20,11 @@ Next, it looked through various local databases for the user's Discord token.

![Code from malware scanning local dbs for Discord tokens](images/pypi_discordcmd_2.png)

It grabbed the data associated with the token from the Discord API and exfiltrated it back to a Discord server controlled by the attacker.
It grabbed the data associated with the token from the Discord API and exfiltrated it back to a Discord server controlled by the attacker.

### NPM: colorsss

*2022-03-05*, [Analysis Result](https://storage.googleapis.com/ossf-malware-analysis-results/npm/colorsss/0.0.2.json)
_2022-03-05_, [Analysis Result](https://storage.googleapis.com/khulnasoft-malware-analysis-results/npm/colorsss/0.0.2.json)

Similar to discordcmd above, this NPM package attempts to steal a Windows user's Discord account token and was discovered by identifying calls to the Discord API. This package:

Expand All @@ -34,11 +34,11 @@ Similar to discordcmd above, this NPM package attempts to steal a Windows user's

## Remote Shell

A remote shell is used by an attacker to provide access to a [command shell](https://en.wikipedia.org/wiki/Shell_(computing)) running on a target machine over the network. These are usually "reverse shells" that connect back to an attacker controlled machine.
A remote shell is used by an attacker to provide access to a [command shell](<https://en.wikipedia.org/wiki/Shell_(computing)>) running on a target machine over the network. These are usually "reverse shells" that connect back to an attacker controlled machine.

### NPM: @roku-web-core/ajax

*2022-03-08*, [Analysis Result](https://storage.googleapis.com/ossf-malware-analysis-results/npm/%40roku-web-core/ajax/1.100000000000003.100000000000003.json)
_2022-03-08_, [Analysis Result](https://storage.googleapis.com/khulnasoft-malware-analysis-results/npm/%40roku-web-core/ajax/1.100000000000003.100000000000003.json)

During install, this NPM package exfiltrates details of the machine it is running on, and then opens a reverse shell, allowing the remote execution of commands.

Expand All @@ -48,7 +48,7 @@ This package was discovered from its requests to an attacker-controlled address.

### PyPI: secrevthree

*2022-03-14*, [Analysis Result](https://storage.googleapis.com/ossf-malware-analysis-results/pypi/secrevthree/0.0.2.json)
_2022-03-14_, [Analysis Result](https://storage.googleapis.com/khulnasoft-malware-analysis-results/pypi/secrevthree/0.0.2.json)

This package opens a simple reverse shell when a specific module is imported from the library, allowing remote command execution. It uses a simple obfuscation technique to make it harder for a reader to notice.

Expand All @@ -58,7 +58,7 @@ This package was discovered from the request to the attacker-controlled address.

### NPM: random-vouchercode-generator

*2022-03-14*, [Analysis Result](https://storage.googleapis.com/ossf-malware-analysis-results/npm/random-vouchercode-generator/1.0.3.json)
_2022-03-14_, [Analysis Result](https://storage.googleapis.com/khulnasoft-malware-analysis-results/npm/random-vouchercode-generator/1.0.3.json)

When the library is imported it queries a server running on the Heroku cloud platform. The server response includes a command and a flag indicating whether or not the command should be run.

Expand All @@ -79,5 +79,5 @@ The vast majority of the malicious packages we detect are [dependency confusion]
The packages found usually contain a simple script that runs during install and calls home with a few details about the host. These packages are most likely the work of security researchers looking for bug bounties, since most are not exfiltrating meaningful data except the name of the machine or a username, and they make no attempt to disguise their behavior.

These dependency confusion attacks were discovered through the domains they used, such as burpcollaborator.net, pipedream.com, interact.sh, which are commonly used for reporting back attacks. The same domains appear across unrelated packages and have no apparent connection to the packages themselves. Many packages also used unusual version numbers that were high (e.g. v5.0.0, v99.10.9) for a package with no previous versions.
![Table showing malicious npm package names, versions and host accessed](images/npm_depconf-typosquat_1.png)

![Table showing malicious npm package names, versions and host accessed](images/npm_depconf-typosquat_1.png)
4 changes: 2 additions & 2 deletions function/loader/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ gcloud functions deploy load-data \
--runtime=go121 \
--timeout=120s \
--trigger-topic=load-data \
--set-env-vars=OSSF_MALWARE_ANALYSIS_RESULTS=ossf-malware-analysis-results,GCP_PROJECT=khulnasoft-malware-analysis
--set-env-vars=KHULNASOFT_MALWARE_ANALYSIS_RESULTS=khulnasoft-malware-analysis-results,GCP_PROJECT=khulnasoft-malware-analysis
```

## Static analysis results
Expand All @@ -33,5 +33,5 @@ gcloud functions deploy load-staticanalysis-data \
--runtime=go121 \
--timeout=120s \
--trigger-topic=load-data \
--set-env-vars=OSSF_MALWARE_STATIC_ANALYSIS_RESULTS=ossf-malware-static-analysis-results-v1,GCP_PROJECT=khulnasoft-malware-analysis
--set-env-vars=KHULNASOFT_MALWARE_STATIC_ANALYSIS_RESULTS=khulnasoft-malware-static-analysis-results-v1,GCP_PROJECT=khulnasoft-malware-analysis
```
4 changes: 2 additions & 2 deletions function/loader/load.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ func runAndWaitForJob(ctx context.Context, loader *bigquery.Loader) error {

func Load(ctx context.Context, m PubSubMessage) error {
project := os.Getenv("GCP_PROJECT")
bucket := os.Getenv("OSSF_MALWARE_ANALYSIS_RESULTS")
bucket := os.Getenv("KHULNASOFT_MALWARE_ANALYSIS_RESULTS")

bq, err := bigquery.NewClient(ctx, project)
if err != nil {
Expand Down Expand Up @@ -77,7 +77,7 @@ func Load(ctx context.Context, m PubSubMessage) error {

func LoadStaticAnalysis(ctx context.Context, m PubSubMessage) error {
project := os.Getenv("GCP_PROJECT")
bucket := os.Getenv("OSSF_MALWARE_STATIC_ANALYSIS_RESULTS")
bucket := os.Getenv("KHULNASOFT_MALWARE_STATIC_ANALYSIS_RESULTS")

bq, err := bigquery.NewClient(ctx, project)
if err != nil {
Expand Down
24 changes: 12 additions & 12 deletions infra/cloudbuild/dynamic_loader/cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
steps:
- name: gcr.io/google.com/cloudsdktool/cloud-sdk
env:
- 'PROJECT_ID=khulnasoft-malware-analysis'
- 'LOAD_DATASET=loading'
- 'LOAD_TABLE_PREFIX=merge_'
- 'DEST_DATASET=packages'
- 'DEST_TABLE=analysis'
- 'RESULT_BUCKET=gs://ossf-malware-analysis-results'
- 'SCHEMA_FILE=function/loader/dynamic-analysis-schema.json'
entrypoint: '/bin/bash'
args: ['scripts/bq_load.sh']
timeout: 43200s # 12 hours
- name: gcr.io/google.com/cloudsdktool/cloud-sdk
env:
- "PROJECT_ID=khulnasoft-malware-analysis"
- "LOAD_DATASET=loading"
- "LOAD_TABLE_PREFIX=merge_"
- "DEST_DATASET=packages"
- "DEST_TABLE=analysis"
- "RESULT_BUCKET=gs://khulnasoft-malware-analysis-results"
- "SCHEMA_FILE=function/loader/dynamic-analysis-schema.json"
entrypoint: "/bin/bash"
args: ["scripts/bq_load.sh"]
timeout: 43200s # 12 hours
options:
logging: CLOUD_LOGGING_ONLY
2 changes: 1 addition & 1 deletion infra/terraform/analysis.tf
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ provider "google" {

terraform {
backend "gcs" {
bucket = "ossf-analysis-tf-state"
bucket = "khulnasoft-analysis-tf-state"
prefix = "terraform/state"
}
}
Expand Down
4 changes: 2 additions & 2 deletions infra/terraform/docker_registry/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ resource "google_artifact_registry_repository" "gcr_docker" {
project = var.project
location = "us"
repository_id = "gcr.io"
description = "gcr.io docker container registry for OSSF Malware Analysis Images"
description = "gcr.io docker container registry for KHULNASOFT Malware Analysis Images"
format = "DOCKER"
}

Expand All @@ -14,7 +14,7 @@ resource "google_artifact_registry_repository" "us_gcr_docker" {
project = var.project
location = "us"
repository_id = "us.gcr.io"
description = "us.gcr.io docker container registry for OSSF Malware Analysis Images"
description = "us.gcr.io docker container registry for KHULNASOFT Malware Analysis Images"
format = "DOCKER"
}

Expand Down
Loading
Loading