diff --git a/docs/code_insights/use-cases.mdx b/docs/code_insights/use-cases.mdx new file mode 100644 index 000000000..bbfb1b40d --- /dev/null +++ b/docs/code_insights/use-cases.mdx @@ -0,0 +1,474 @@ +# Common Code Insights use cases and recipes + +

This section highlights common use cases for Code Insights along with example data series queries to help you get started.

+ +You can [filter repositories in real time](/code_insights/how-tos/filtering_an_insight) or add any [Sourcegraph search filter](https://sourcegraph.com/docs/code-search/queries/language#search-pattern) to your data series query — including filters for language, directory, or content — to refine your insights. + +Sample queries using `commit` and `diff` searches currently work **only** with Code Insights that run over **explicit lists of specific repositories**. + +The sample queries below **exclude [forked and archived repositories](/code_insights/references/common_reasons_code_insights_may_not_match_search_results#not-including-fork-no-and-archived-no-in-your-insight-query) by default**. If you want to include them, you can manually add the appropriate flags to your queries. + +## Popular + +Quick insights that are widely used to measure version usage, language trends, and best practices across repositories. + +### Terraform versions + +Detect and track which Terraform versions are present or most popular in your codebase +```sgquery +app.terraform.io/(.*)\n version =(.*)1.1.0 patternType:regexp lang:Terraform +``` +```sgquery +app.terraform.io/(.*)\n version =(.*)1.2.0 patternType:regexp lang:Terraform +``` + +### Global CSS to CSS modules + +Tracking migration from global CSS to CSS modules +```sgquery +select:file lang:SCSS -file:module patterntype:regexp +``` +```sgquery +select:file lang:SCSS file:module patterntype:regexp +``` + +### Vulnerable and fixed Log4j versions + +Confirm the removal of vulnerable log4j versions and ensure only fixed versions remain. +```sgquery +lang:gradle org\.apache\.logging\.log4j['"] 2\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16)(\.[0-9]+) patterntype:regexp +``` +```sgquery +lang:gradle org\.apache\.logging\.log4j['"] 2\.(17)(\.[0-9]+) patterntype:regexp +``` + +### Yarn adoption + +Are more repos increasingly using yarn? Track yarn adoption across teams and groups in your organization +```sgquery +select:repo file:yarn.lock +``` + +### Java versions + +Detect and track which Java versions are most popular in your codebase + +*Uses the [detect and track](/code_insights/explanations/automatically_generated_data_series) capture groups insight type* +```sgquery +file:pom\.xml$ (.*) +``` + +### Linter override rules + +A code health indicator for how many linter override rules exist +```sgquery +file:^\.eslintignore .\n patternType:regexp +``` + +### Language use over time + +Track the growth of certain languages by file count +```sgquery +select:file lang:TypeScript +``` +```sgquery +select:file lang:JavaScript +``` + +### Pinned vs Unpinned Docker Base Images + +Track how many unpinned images exist relative to pinned images +```sgquery +^FROM (\w+\/)?\w+:latest($|\s) file:Dockerfile patternType:regexp +``` +```sgquery +^FROM (\w+\/)?\w+:latest@sha file:Dockerfile patternType:regexp +``` + +## Migration + +Track the progress of large-scale code migrations or refactors. + +### Config or docs file + +How many repos contain a config or docs file in a specific directory +```sgquery +select:repo file:docs/*/new_config_filename +``` + +### “blacklist/whitelist” to “denylist/allowlist” + +How the switch from files containing “blacklist/whitelist” to “denylist/allowlist” is progressing +```sgquery +select:file blacklist OR whitelist +``` +```sgquery +select:file denylist OR allowlist +``` + +### Global CSS to CSS modules + +Tracking migration from global CSS to CSS modules +```sgquery +select:file lang:SCSS -file:module patterntype:regexp +``` +```sgquery +select:file lang:SCSS file:module patterntype:regexp +``` + +### Python 2 to Python 3 + +Track progress of the major Python version migration. +```sgquery +#!/usr/bin/env python3 +``` +```sgquery +#!/usr/bin/env python2 +``` + +### React Class to Function Components Migration + +What's the status of migrating to React function components from class components +```sgquery +patternType:regexp const\s\w+:\s(React\.)?FunctionComponent +``` +```sgquery +patternType:regexp extends\s(React\.)?(Pure)?Component +``` + +## Adoption + +Monitor the uptake of tools, frameworks, or libraries in your org. + +### New API usage + +How many repos or teams are using a new API your team built +```sgquery +select:repo ourApiLibraryName.load +``` + +### Yarn adoption + +Are more repos increasingly using yarn? Track yarn adoption across teams and groups in your organization +```sgquery +select:repo file:yarn.lock +``` + +### Frequently used databases + +Which databases we are calling or writing to most often +```sgquery +redis\.set patternType:regexp +``` +```sgquery +graphql\( patternType:regexp +``` + +### Large or expensive package usage + +Understand if a growing number of repos import a large/expensive package +```sgquery +select:repo import\slargePkg patternType:regexp +``` + +### React Component use + +How many places are importing components from a library +```sgquery +from '@sourceLibrary/component' patternType:keyword +``` + +### CI tooling adoption + +How many repos are using our CI system +```sgquery +file:\.circleci/config.yml select:repo +``` + +## Deprecation + +Ensure deprecated patterns, tools, or files are removed. + +### CSS class + +Removal of all deprecated CSS classes +```sgquery +deprecated-class +``` + +### Icon or image + +The removal of all deprecated icon or image instances +```sgquery +2018logo.png +``` + +### Structural code pattern + +Deprecating a structural code pattern in favor of a safer pattern, like how many tries don't have catches +```sgquery +try {:[_]} catch (:[e]) { } finally {:[_]} lang:java patternType:structural +``` + +### Tooling + +The progress of deprecating tooling you’re moving off of +```sgquery +deprecatedEventLogger.log +``` + +### Var keywords + +Number of var keywords in the code base (ES5 deprecation) +```sgquery +(lang:TypeScript OR lang:JavaScript) var ... = patterntype:structural +``` + +### Consolidation of Testing Libraries + +Which React test libraries are being consolidated +```sgquery +from '@testing-library/react' +``` +```sgquery +from 'enzyme' +``` + +## Versions and patterns + +Automatically detect and track patterns or version usage via capture groups. + + These examples are all for use with the [automatically generated data series](/code_insights/explanations/automatically_generated_data_series) of "Detect and track" Code Insights, using regular expression capture groups. + +### Java versions + +Detect and track which Java versions are most popular in your codebase +```sgquery +file:pom\.xml$ (.*) +``` + +### License types in the codebase + +See the breakdown of licenses from package.json files +```sgquery +file:package.json "license":\s"(.*)" +``` + +### All log4j versions + +Which log4j versions are present, including vulnerable versions +```sgquery +lang:gradle org\.apache\.logging\.log4j['"] 2\.([0-9]+)\. +``` + +### Python versions + +Which python versions are in use or haven’t been updated +```sgquery +#!/usr/bin/env python([0-9]\.[0-9]+) +``` + +### Node.js versions + +Which node.js versions are present based on nvm files +```sgquery +nvm\suse\s([0-9]+\.[0-9]+) +``` + +### CSS Colors + +What CSS colors are present or most popular +```sgquery +color:#([0-9a-fA-f]{3,6}) +``` + +### Types of checkov skips + +Identify the most common reasons security checks in checkov are skipped +```sgquery +patterntype:regexp file:.tf #checkov:skip=(.*) +``` + +### Tracer calls + +See all your tracer calls to minimize spend on, or track the growth of, tools like Datadog. +```sgquery +tracer\.trace\(([\s"'\w@\/:^.#,+-=]+)\) +``` + +## Code health + +Monitor trends in maintainability, documentation coverage, and overall code hygiene. + +### TODOs + +How many TODOs are in a specific part of the codebase (or all of it) +```sgquery +TODO +``` + +### Linter override rules + +A code health indicator for how many linter override rules exist +```sgquery +file:^\.eslintignore .\n patternType:regexp +``` + +### Commits with “revert” + +How frequently there are commits with “revert” in the commit message +```sgquery +type:commit revert +``` + +### Deprecated calls + +How many times deprecated calls are used +```sgquery +lang:java @deprecated +``` + +### Storybook tests + +Track the number of existing Storybook tests +```sgquery +patternType:regexp f:\.story\.tsx$ \badd\( +``` + +### Repos with Documentation + +How many repos do or don't have READMEs +```sgquery +repohasfile:readme select:repo +``` +```sgquery +-repohasfile:readme select:repo +``` + +### Ownership via CODEOWNERS files + +How many repos do or don't have CODEOWNERS files +```sgquery +repohasfile:CODEOWNERS select:repo +``` +```sgquery +-repohasfile:CODEOWNERS select:repo +``` + +### CI tooling adoption + +How many repos are using our CI system +```sgquery +file:\.circleci/config.yml select:repo +``` + +## Security + +Track security regressions, key leaks, and library deprecations. + +### Vulnerable open source library + +Confirm that a vulnerable open-source library has been fully removed, or track the speed of its deprecation +```sgquery +vulnerableLibrary@14.3.9 +``` + +### API keys + +How quickly we notice and remove API keys when they are committed +```sgquery +regexMatchingAPIKey patternType:regexp +``` + +### Vulnerable and fixed Log4j versions + +Confirm that vulnerable versions of log4j are removed and only fixed versions appear +```sgquery +lang:gradle org\.apache\.logging\.log4j['"] 2\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16)(\.[0-9]+) patterntype:regexp +``` +```sgquery +lang:gradle org\.apache\.logging\.log4j['"] 2\.(17)(\.[0-9]+) patterntype:regexp +``` + +### How many tests are skipped + +See how many tests have skip conditions +```sgquery +(this.skip() OR it.skip) lang:TypeScript +``` + +### Tests amount and types + +See what types of tests are most common and total counts +```sgquery +patternType:regexp case:yes \b(it|test)\( f:/end-to-end/.*\.test\.ts$ +``` +```sgquery +patternType:regexp case:yes \b(it|test)\( f:/regression/.*\.test\.ts$ +``` +```sgquery +patternType:regexp case:yes \b(it|test)\( f:/integration/.*\.test\.ts$ +``` + +### Types of checkov skips + +See the most common reasons for why secuirty checks in checkov are skipped + +*Uses the [detect and track](/code_insights/explanations/automatically_generated_data_series) capture groups insight type* +```sgquery +patterntype:regexp file:.tf #checkov:skip=(.*) +``` + +## Other + +### Typescript vs. Go + +Are there more Typescript or more Go files +```sgquery +select:file lang:TypeScript +``` +```sgquery +select:file lang:Go +``` + +### iOS app screens + +What number of iOS app screens are in the entire app +```sgquery +struct\s(.*):\sview$ patternType:regexp lang:swift +``` + +### Adopting new API by Team + +Which teams or repos have adopted a new API so far +```sgquery +file:mobileTeam newAPI.call +``` +```sgquery +file:webappTeam newAPI.call +``` +*Or [filter teams by repositories](/code_insights/how-tos/filtering_an_insight) in real time* + +### Problematic API by Team + +Which teams have the most usage of a problematic API +```sgquery +problemAPI file:teamOneDirectory +``` +```sgquery +problemAPI file:teamTwoDirectory +``` +*Or [filter teams by repositories](/code_insights/how-tos/filtering_an_insight) in real time* + +### Data fetching from GraphQL + +What GraphQL operations are being called often +```sgquery +patternType:regexp requestGraphQL(\(|<[^>]*>\() +``` +```sgquery +patternType:regexp (query|mutate)GraphQL(\(|<[^>]*>\() +``` +```sgquery +patternType:regexp use(Query|Mutation|Connection|LazyQuery)(\(|<[^>]*>\() +```