Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve qc context definition and reporting #140

Merged
merged 4 commits into from
Aug 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions rinex-cli/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
product/
config/
workspace/
4 changes: 2 additions & 2 deletions rinex-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ This tool supports gzip compressed files but the file name must be terminated by

### Analysis and report files

Reports and plots are rendered in HTML in the `rinex/rinex-cli/product` directory.
Reports and plots are rendered in HTML in the `rinex/rinex-cli/workspace` directory.
Analysis is named after the primary RINEX file.

## `teqc` operations
Expand Down Expand Up @@ -150,7 +150,7 @@ for every operation that did apply (correct command line description).
`> record analysis` means the analysis is starting at this point.
The location of the graphs that were rendered (if any) is given.

[rinex-cli/product](product/) is where all analysis reports
[rinex-cli/workspace](workspace/) is where all analysis reports
get generated. It is named after the main RINEX file (`-fp`) which
allows preserving sessions for different files.

Expand Down
8 changes: 3 additions & 5 deletions rinex-cli/doc/qc.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,9 @@ This allows for example Glonass and other NAV context to be correctly defined.
1. `--fp [FILE]` for the Observation file
2. `--nav [FILE1] [FILE2]..` : pass NAV contexts

We will generate products in a dedicated folder.

The current behavior is to use the
[product](https://github.com/gwbres/rinex/tree/rinex-cli/product)
folder to generate QC reports.
We will generate products the
[workspace](https://github.com/gwbres/rinex/tree/rinex-cli/workspace)
folder, that includes QC reports.

Unlike teqc, we do not support BINEX nor SP3 input data/files as of today.

Expand Down
11 changes: 6 additions & 5 deletions rinex-cli/src/analysis/sampling.rs
Original file line number Diff line number Diff line change
@@ -1,28 +1,29 @@
use crate::{plot::PlotContext, Context};
use crate::plot::PlotContext;
use itertools::Itertools;
use plotly::Histogram; //.sorted()
use rinex::quality::QcContext;

/*
* Sampling histogram
*/
pub fn histogram(ctx: &Context, plot_ctx: &mut PlotContext) {
pub fn histogram(ctx: &QcContext, plot_ctx: &mut PlotContext) {
plot_ctx.add_cartesian2d_plot("Sampling Histogram", "Count");
let durations: Vec<_> = ctx
.primary_rinex
.primary_data()
.sampling_histogram()
.sorted()
.map(|(dt, _)| dt.to_string())
.collect();
let populations: Vec<_> = ctx
.primary_rinex
.primary_data()
.sampling_histogram()
.sorted()
.map(|(_, pop)| pop.to_string())
.collect();
let histogram = Histogram::new_xy(durations, populations).name("Sampling Histogram");
plot_ctx.add_trace(histogram);

if let Some(ref nav) = ctx.nav_rinex {
if let Some(nav) = &ctx.navigation_data() {
// Run similar analysis on NAV context
let durations: Vec<_> = nav
.sampling_histogram()
Expand Down
25 changes: 12 additions & 13 deletions rinex-cli/src/analysis/sv_epoch.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,18 @@
use crate::{
plot::{build_chart_epoch_axis, generate_markers, PlotContext},
Context,
};
use crate::plot::{build_chart_epoch_axis, generate_markers, PlotContext};
use ndarray::Array;
use plotly::common::{Marker, Mode, Title, Visible};
use plotly::layout::Axis;
use rinex::prelude::*;
use rinex::{prelude::*, quality::QcContext};

/*
* Sv per epoch analysis
*/
pub fn sv_epoch(ctx: &Context, plot_ctx: &mut PlotContext) {
pub fn sv_epoch(ctx: &QcContext, plot_ctx: &mut PlotContext) {
plot_ctx.add_cartesian2d_plot("Sv per Epoch", "Sv(PRN#)");
/*
* plot customization
* dy axis: 1.0, since we're plotting PRN# here
* We're plotting PRN#, set dy to +/- 1
* for nicer rendition
*/
let plot_item = plot_ctx.plot_item_mut().unwrap();
let layout = plot_item.layout().clone().y_axis(
Expand All @@ -25,19 +23,20 @@ pub fn sv_epoch(ctx: &Context, plot_ctx: &mut PlotContext) {
);
plot_item.set_layout(layout);

// markers/symbols: one per constellation system
let constellations: Vec<_> = ctx.primary_rinex.constellation().collect();
// Design markers / symbols
// one per constellation system
let constellations: Vec<_> = ctx.primary_data().constellation().collect();
let mut nb_markers = constellations.len();

if let Some(ref nav) = ctx.nav_rinex {
if let Some(ref nav) = ctx.navigation_data() {
nb_markers += nav.constellation().count();
}

let markers = generate_markers(nb_markers);

let data: Vec<_> = ctx.primary_rinex.sv_epoch().collect();
let data: Vec<_> = ctx.primary_data().sv_epoch().collect();

for (sv_index, sv) in ctx.primary_rinex.sv().enumerate() {
for (sv_index, sv) in ctx.primary_data().sv().enumerate() {
let epochs: Vec<Epoch> = data
.iter()
.filter_map(|(epoch, ssv)| {
Expand Down Expand Up @@ -68,7 +67,7 @@ pub fn sv_epoch(ctx: &Context, plot_ctx: &mut PlotContext) {
plot_ctx.add_trace(trace);
}

if let Some(ref nav) = ctx.nav_rinex {
if let Some(nav) = &ctx.navigation_data() {
let data: Vec<_> = nav.sv_epoch().collect();
let nav_constell: Vec<_> = nav.constellation().collect();

Expand Down
120 changes: 42 additions & 78 deletions rinex-cli/src/cli.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
use crate::fops::filename;
use crate::parser::parse_epoch;
use clap::{Arg, ArgAction, ArgMatches, ColorChoice, Command};
use log::{error, info, warn};
use rinex::{prelude::*, quality::QcOpts, Merge};
use log::{error, info};
use rinex::{prelude::*, quality::QcOpts};
use std::path::Path;
use std::str::FromStr;

pub struct Cli {
Expand Down Expand Up @@ -254,19 +253,8 @@ Refer to README"))
self.matches.get_one::<String>("filepath").unwrap() // mandatory flag
}
/// Returns output filepaths
pub fn output_path(&self) -> Option<&str> {
if let Some(args) = self.matches.get_one::<String>("output") {
Some(&args)
} else {
None
}
}
pub fn mask_filters(&self) -> Vec<&String> {
if let Some(filters) = self.matches.get_many::<String>("mask-filter") {
filters.collect()
} else {
Vec::new()
}
pub fn output_path(&self) -> Option<&String> {
self.matches.get_one::<String>("output")
}
pub fn preprocessing(&self) -> Vec<&String> {
if let Some(filters) = self.matches.get_many::<String>("preprocessing") {
Expand All @@ -287,16 +275,24 @@ Refer to README"))
}
pub fn qc_config(&self) -> QcOpts {
if let Some(path) = self.qc_config_path() {
let s = std::fs::read_to_string(path).expect(&format!("failed to read \"{}\"", path));
let opts: QcOpts = serde_json::from_str(&s).expect("faulty qc configuration");
opts
if let Ok(content) = std::fs::read_to_string(path) {
let opts = serde_json::from_str(&content);
if let Ok(opts) = opts {
opts
} else {
error!("failed to parse parameter file \"{}\"", path);
info!("using default parameters");
QcOpts::default()
}
} else {
error!("failed to read parameter file \"{}\"", path);
info!("using default parameters");
QcOpts::default()
}
} else {
QcOpts::default()
}
}
pub fn quality_check_separate(&self) -> bool {
self.matches.get_flag("qc-separate")
}
pub fn quality_check_only(&self) -> bool {
self.matches.get_flag("qc-only")
}
Expand Down Expand Up @@ -390,30 +386,39 @@ Refer to README"))
pub fn quiet(&self) -> bool {
self.matches.get_flag("quiet")
}
pub fn tiny_html(&self) -> bool {
self.matches.get_flag("tiny-html")
}
pub fn cs_graph(&self) -> bool {
self.matches.get_flag("cs")
}
/*
* Returns possible file path to merge
*/
pub fn merge_path(&self) -> Option<&Path> {
self.matches
.get_one::<String>("merge")
.and_then(|s| Some(Path::new(s)))
}
/// Returns optionnal RINEX file to "merge"
pub fn to_merge(&self) -> Option<Rinex> {
let fp = self.matches.get_one::<String>("merge")?;
if let Ok(rnx) = Rinex::from_file(&fp) {
Some(rnx)
if let Some(path) = self.merge_path() {
let path = path.to_str().unwrap();
if let Ok(rnx) = Rinex::from_file(path) {
Some(rnx)
} else {
error!("failed to parse \"{}\"", path);
None
}
} else {
error!("failed to parse \"{}\"", filename(fp));
None
}
}
/// Returns split operation args
pub fn split(&self) -> Option<Epoch> {
if self.matches.contains_id("split") {
if let Some(args) = self.matches.get_one::<String>("split") {
if let Ok(epoch) = parse_epoch(args) {
if let Some(s) = self.matches.get_one::<String>("split") {
if let Ok(epoch) = Epoch::from_str(s) {
Some(epoch)
} else {
panic!("failed to parse [DATETIME]");
panic!("failed to parse [EPOCH]");
}
} else {
None
Expand All @@ -423,52 +428,11 @@ Refer to README"))
}
}
/// Returns optionnal Nav path, for enhanced capabilities
pub fn nav_paths(&self) -> Vec<&String> {
if let Some(paths) = self.matches.get_many::<String>("nav") {
paths.collect()
} else {
Vec::new()
}
}
/// Returns optionnal Navigation context
pub fn nav_context(&self) -> Option<Rinex> {
let mut nav_ctx: Option<Rinex> = None;
let paths = self.nav_paths();
for path in paths {
if let Ok(rnx) = Rinex::from_file(&path) {
if let Some(ref mut ctx) = nav_ctx {
let _ = ctx.merge_mut(&rnx);
} else {
trace!("(nav) augmentation: \"{}\"", filename(&path));
nav_ctx = Some(rnx);
}
} else {
error!("failed to parse navigation file \"{}\"", filename(&path));
}
}
nav_ctx
}
fn atx_path(&self) -> Option<&String> {
if self.matches.contains_id("atx") {
self.matches.get_one::<String>("atx")
} else {
None
}
pub fn nav_path(&self) -> Option<&String> {
self.matches.get_one::<String>("nav")
}
pub fn atx_context(&self) -> Option<Rinex> {
if let Some(path) = self.atx_path() {
if let Ok(rnx) = Rinex::from_file(&path) {
if rnx.is_antex_rinex() {
info!("--atx context provided");
return Some(rnx);
} else {
warn!("--atx should be antenna rinex file");
}
} else {
error!("failed to parse atx file \"{}\"", filename(&path));
}
}
None
pub fn atx_path(&self) -> Option<&String> {
self.matches.get_one::<String>("atx")
}
fn manual_ecef(&self) -> Option<&String> {
self.matches.get_one::<String>("antenna-ecef")
Expand Down
69 changes: 0 additions & 69 deletions rinex-cli/src/context.rs

This file was deleted.

Loading