Skip to content

Commit

Permalink
feat(#28): count of lines in script (#88)
Browse files Browse the repository at this point in the history
- Add script to call cloc
- Update dry-run
- Improve README

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
  • Loading branch information
pylapp authored Jul 18, 2023
1 parent d7d4e6c commit ba1bd99
Show file tree
Hide file tree
Showing 3 changed files with 208 additions and 2 deletions.
46 changes: 45 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

Toolbox to help developers and open source referents to have cleaner projects in _GitHub_ organizations, and more.

Toolbox is mainly written in _Shell_ because this language is very efficient for files processing and provides a strong and rich standard API with cool primitives. Contains also _Ruby_ scripts. _Ruby_ are shiny gems, I love them. _Python_ is also used.
Toolbox is mainly written in _Shell_ because this language is very efficient for files processing and provides a strong and rich standard API with cool primitives. It helps also to call system primitives easily. Contains also _Ruby_ scripts. _Ruby_ are shiny gems, I love them. _Python_ is also used.

# Environment

Expand Down Expand Up @@ -100,6 +100,22 @@ Table of Contents

# The "diver" of source code and commits

## Prerequisites

Some components must be available (dry run script will help to find the ones missing):
- Ruby (version 2.7.1)
- Bash (version 3.2.5)
- Python3 (version 3.8.5)
- Git (version 2.32.0)
- GH (version 1.31.1)
- Gitleaks (version 8.3.0)
- Octokit Ruby gem (version 4.20.0)
- Cloc (version 1.88)

This project expects to have this tools available and already added in your system.
Thus there is not composition nor aggregation (mere or not) about them.
None of them have been modified nor distributed.

## File of words

Some features parse a text file containing the words to look for in sources or git logs.
Expand Down Expand Up @@ -255,6 +271,17 @@ bash extract-emails-from-history.sh --project path/to.git/project --loglimit gi
The log limit is the value to pass to the `git log` command, e.g. _2.weeks_ or _3.years_.
_path/to/the/project_ points to the root of the project to analyse.

### Count lines of code in a directory

_Keywords: #cloc #metrics #KPI #APP_

You can use one script to compute lines of code thanks to [cloc](https://github.com/AlDanial/cloc) program.
It will generate a report with all the metrics you may need.

```bash
bash lines-count.sh --target "absolute/path/to/target"
```

# Play with GitHub web API

## Prerequisites
Expand All @@ -267,6 +294,10 @@ _path/to/the/project_ points to the root of the project to analyse.
- Define the _GitHub_ organization name in the _configuration.rb_ file for the `GITHUB_ORGANIZATION_NAME` variable. It will allow to send requests to query and modify your organization.
- Define also the logins of the GitHub adminsitrators of your organization so as to prevent to change their permisssion for example.

This project expects to have this tools available and already added in your system.
Thus there is not composition nor aggregation (mere or not) about them.
None of them have been modified nor distributed.

## Prepare project

```shell
Expand Down Expand Up @@ -437,6 +468,10 @@ Some configuration details must be defined (in _configuration.rb), like:
2. `REPOSITORIES_CLONE_LOCATION_PATH` location of the clone sif you want to make a dump of the organisation repositories
3. `REPOSITORIES_CLONE_URL_JSON_KEY` to choose the JSON key to get the repository URL from GitHub API

This project expects to have this tools available and already added in your system.
Thus there is not composition nor aggregation (mere or not) about them.
None of them have been modified nor distributed.

## Prepare project

```shell
Expand Down Expand Up @@ -525,6 +560,10 @@ You need to define in the _configuration.rb_ files the Github organisation at **
- Create a [GitLab personal token](https://gitlab.com/-/profile/personal_access_tokens) and define it in the _configuration.rb_ file for the `GILAB_PERSONAL_ACCESS_TOKEN` variable.
- Define the GitLab organization id in the _configuration.rb_ file for the `GITLAB_ORGANIZATION_ID` variable. It will allow to send requests to query and modify your organization.

This project expects to have this tools available and already added in your system.
Thus there is not composition nor aggregation (mere or not) about them.
None of them have been modified nor distributed.

## Prepare project

```ruby
Expand Down Expand Up @@ -603,6 +642,11 @@ pip install pytest
pip install beautifulsoup4
```

This project expects to have this tools available and already added in your system.
None of them have been modified nor distributed.
It could be seen as a [mere aggregation](https://www.gnu.org/licenses/gpl-faq.html#MereAggregation) of components, you can get licenses details in THIRD-PARTY.txt and licenses folder.


## Fill the configuration file

Before to use the tools, the file 'config.ini' is at the root of the project, you have to personalize this file.
Expand Down
6 changes: 5 additions & 1 deletion toolbox/diver/dry-run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

# Since...............: 10/03/2023
# Description.........: Make a dry-run of the diver module to check if everything is ready to use
# Version.............: 1.0.0
# Version.............: 1.1.0

set -eu

Expand Down Expand Up @@ -112,6 +112,7 @@ CheckIfFileExists "./find-credentials-in-files.sh"
CheckIfFileExists "./find-credits-in-files.sh"
CheckIfFileExists "./find-missing-developers-in-git-commits.sh"
CheckIfFileExists "./list-contributors-in-history.sh"
CheckIfFileExists "./lines-count.sh"

echo -e "\nCheck utilitary scripts..."
CheckIfFileExists "./utils/extract-contributors-lists.rb"
Expand Down Expand Up @@ -150,6 +151,9 @@ CheckIfRuntimeExists "Gitleaks" "gitleaks version" "8.3.0"
echo -e "\nCheck for Octokit..."
CheckIfRuntimeExists "Octokit (Ruby gem)" "gem list | grep octokit" "4.20.0"

echo -e "\nCheck for cloc..."
CheckIfRuntimeExists "cloc" "cloc --version" "1.88"

# Conclusion
# ----------

Expand Down
158 changes: 158 additions & 0 deletions toolbox/diver/lines-count.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
#!/bin/bash
# Software Name: floss-toolbox
# SPDX-FileCopyrightText: Copyright (c) 2020-2023 Orange
# SPDX-License-Identifier: Apache-2.0
#
# This software is distributed under the Apache 2.0 license.
#
# Author: Pierre-Yves LAPERSONNE <pierreyves(dot)lapersonne(at)orange(dot)com> et al.

# Version.............: 1.0.0
# Since...............: 18/07/2023
# Description.........: Counts liens of cound for a target directory
#
# Usage: bash lines-count.sh --target TARGET
#
# Exit codes:
# 0 - normal exit
# 1 - problem with given parameters
#

set -euo pipefail # set -euxo pipefail

VERSION="1.0.0"
SCRIPT_NAME="lines-count.sh"

# -------------
# Configuration
# -------------

NORMAL_EXIT_CODE=0
BAD_ARGUMENTS_EXIT_CODE=1

# Folder fo generated files
TEMP_FOLDER="./data"

# Prefix for generated files
GENERATED_FILES_PREFIX="$$-lines-count"

# Report file with metrics
REPORT_METRIC_FILE="$TEMP_FOLDER/$GENERATED_FILES_PREFIX-report.txt"

# ---------
# Functions
# ---------

# \fn DisplayUsages
# \brief Displays an help message and exists
DisplayUsages(){
echo "***********************************************"
echo "$SCRIPT_NAME - Version $VERSION"
echo "***********************************************"
echo "USAGE:"
echo "bash $SCRIPT_NAME --target TARGET"
echo -e "\t --target.........: TARGET must point to a directory to scan for lines and count them (recursive)"
}

# \fn NormalExit
# \brief Exits with NORMAL_EXIT_CODE code
NormalExit(){
exit $NORMAL_EXIT_CODE
}

# \fn BadArgumentsExit
# \brief Exits with BAD_ARGUMENTS_EXIT_CODE code
BadArgumentsExit(){
exit $BAD_ARGUMENTS_EXIT_CODE
}

# \fn BadPreconditionsExit
# \brief Exits with BAD_PRECONDITION_EXIT_CODE code
BadPreconditionsExit() {
exit $BAD_PRECONDITION_EXIT_CODE
}

# \fn CleanFiles
# \brief If existing removes the work files
CleanFiles() {
if [ -f $REPORT_METRIC_FILE ]; then
rm $REPORT_METRIC_FILE
fi
}

# \fn PrepareFiles
# \brief If existing removes the work files and then creates them
PrepareFiles() {
CleanFiles
touch $REPORT_METRIC_FILE
}

# ----------------
# Step 0 - Prepare
# ----------------

# Check the args numbers and display usage if needed
if [ "$#" -ne 2 ]; then
DisplayUsages
NormalExit
fi

# Get target
if [ "$1" = "--target" ]; then
if [ "$2" ]; then
directory_to_scan=$2
else
DisplayUsages
BadArgumentsExit
fi
else
DisplayUsages
BadArgumentsExit
fi

# Check if target is directory
if [ ! -d "$directory_to_scan" ]; then
echo "💥 Error: Target is not a directory ($directory_to_scan)."
BadArgumentsExit
fi

# Run!
SECONDS=0

echo "**************************************************"
echo "$SCRIPT_NAME - Version $VERSION"
echo "**************************************************"

echo -e "\n"

echo "📋 Directory to scan is to analyse is $directory_to_scan"
echo "📋 Prepare logs"
PrepareFiles

echo -e "\n"

# ------------------------------------------------
# Step 1 - Count lines of target and store results
# ------------------------------------------------

echo "🍥 Counting lines and store in $directory_to_scan"

cloc "$directory_to_scan" > "$REPORT_METRIC_FILE"

echo -e "👌 Counting of lines is done\n"

# ------------------------------------------------------------
# Step 2 - Metrics (words and files counts, hits, duration...)
# ------------------------------------------------------------

script_duration=$SECONDS
echo "📈 Elapsed time.............................................: $(($script_duration / 60))' $(($script_duration % 60))''"

# The end!

echo -e "\nReport available in $REPORT_METRIC_FILE:"
cat $REPORT_METRIC_FILE

echo -e "\nEnd of $SCRIPT_NAME\n"
CleanFiles
NormalExit

0 comments on commit ba1bd99

Please sign in to comment.