Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adaptation to 96TS in import NTC2 #207

Merged
merged 2 commits into from
Dec 4, 2024
Merged

Conversation

kahyami
Copy link
Collaborator

@kahyami kahyami commented Dec 3, 2024

Please check if the PR fulfills these requirements (please use '[x]' to check the checkboxes, or submit the PR and then click the checkboxes)

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

Does this PR already have an issue describing the problem ? If so, link to this issue using '#XXX' and skip the rest

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

What is the current behavior? (You can also link to an open issue here)

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

Other information:

(if any of the questions/checkboxes don't apply, please delete them entirely)

Summary by CodeRabbit

  • New Features

    • Enhanced logic for determining position in time series data based on size, improving error handling for unexpected data sizes.
    • New XML files added to represent capacity documents with detailed metadata and structured time series data.
  • Bug Fixes

    • Updated tests to ensure proper validation of import functionality and error handling for missing intervals.
  • Tests

    • Introduced new test cases to validate import functionality and updated existing tests to reflect changes in validation logic.

Copy link

coderabbitai bot commented Dec 3, 2024

Walkthrough

The changes involve significant updates to the Ntc2Util.java class, specifically the getD2ExchangeByOffsetDateTime method, which now includes enhanced logic for calculating the position based on the size of the qtyByPositionMap. Additionally, new XML files have been introduced to support testing scenarios, including cases with complete and missing intervals. Test cases in FileImporterTest have been updated to reflect these changes, improving validation and error handling.

Changes

File Change Summary
.../Ntc2Util.java Updated getD2ExchangeByOffsetDateTime method to include a new variable for CET time zone, modified logic for calculating position based on qtyByPositionMap size, and simplified return statement.
.../FileImporterTest.java Added testImportNtc296TSAllPresent method to verify import functionality with specific XML files; modified testImportFailsWithMissingPositions to use new XML files and updated expected exception messages.
.../services/NTC2_20210624_2D5_AT-IT1-test-96.xml New XML file defining a CapacityDocument structure with various metadata elements and a time series of intervals.
.../services/NTC2_20210624_2D5_AT-IT1-test-missing-intervals.xml New XML file containing a structured representation of a capacity document with missing intervals.
.../services/NTC2_20210624_2D5_AT-IT1-test.xml Modified to include 24 new <Interval> elements, expanding the capacity time series data.
.../services/NTC2_20210624_2D5_CH-IT1-test-96.xml New XML file defining a CapacityDocument structure with metadata and a series of intervals.
.../services/NTC2_20210624_2D5_CH-IT1-test-missing-intervals.xml New XML file with a structured capacity document representing missing intervals.
.../services/NTC2_20210624_2D5_CH-IT1-test.xml Modified to include 24 new <Interval> elements, enhancing the time series data.
.../services/NTC2_20210624_2D5_FR-IT1-test-96.xml New XML file defining a CapacityDocument structure with metadata and intervals.
.../services/NTC2_20210624_2D5_FR-IT1-test-missing-intervals.xml New XML file representing a CapacityDocument with missing intervals.
.../services/NTC2_20210624_2D5_FR-IT1-test.xml Modified to include 24 new <Interval> elements, expanding the capacity time series data.
.../services/NTC2_20210624_2D5_SI-IT1-test-96.xml New XML file representing a CapacityDocument with various nested elements and intervals.
.../services/NTC2_20210624_2D5_SI-IT1-test-missing-intervals.xml New XML file defining a CapacityDocument structure with missing intervals.
.../services/NTC2_20210624_2D5_SI-IT1-test.xml Modified to include 24 new <Interval> elements, enhancing the time series data.

Poem

🐰 In the meadow where data flows,
New XMLs sprout like springtime's rose.
With intervals plenty, and tests that gleam,
Ntc2Util shines, fulfilling the dream.
Error handling's bright, like the sun's warm ray,
Hooray for the changes, hip-hip-hooray! 🌼


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 5308ad3 and 54affe1.

📒 Files selected for processing (1)
  • cse-cc-import-runner-app/src/main/java/com/farao_community/farao/cse/import_runner/app/util/Ntc2Util.java (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • cse-cc-import-runner-app/src/main/java/com/farao_community/farao/cse/import_runner/app/util/Ntc2Util.java

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Outside diff range and nitpick comments (6)
cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test.xml (1)

Line range hint 17-119: Critical: Time interval mismatch with number of intervals

There's an inconsistency in the time intervals:

  • The TimeInterval in Period tag specifies "2021-06-23T23:00Z/2021-06-24T01:00Z" (2-hour period)
  • However, 24 intervals are defined, each representing an hour with PT60M resolution
  • This mismatch could cause issues in production where accurate time mapping is crucial

Fix the TimeInterval to match the number of intervals:

-			<TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
+			<TimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />
cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_SI-IT1-test.xml (1)

Line range hint 12-119: Fix inconsistency between TimeInterval and number of intervals

There's a mismatch between the defined TimeInterval and the number of intervals:

  • TimeInterval specifies "2021-06-23T23:00Z/2021-06-24T01:00Z" (2-hour period)
  • Resolution is "PT60M" (1 hour)
  • 24 intervals are defined

This is inconsistent as a 2-hour period should only contain 2 intervals with the given resolution.

Please either:

  1. Adjust the TimeInterval to match the 24 intervals:
-		<TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
+		<TimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />
  1. Or reduce the number of intervals to match the 2-hour period.
cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test.xml (1)

Line range hint 10-10: Fix inconsistency between TimeInterval and number of intervals

There's a critical mismatch between the declared time interval and the actual data:

  • The CapacityTimeInterval spans 24 hours: 2021-06-23T23:00Z/2021-06-24T23:00Z
  • But the Period/TimeInterval only spans 2 hours: 2021-06-23T23:00Z/2021-06-24T01:00Z
  • Yet the data contains 24 hourly intervals

This inconsistency could cause runtime issues in production.

Update the Period's TimeInterval to match the full 24-hour span:

-			<TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
+			<TimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />

Also applies to: 36-119

cse-cc-import-runner-app/src/main/java/com/farao_community/farao/cse/import_runner/app/util/Ntc2Util.java (1)

52-58: Use constants for interval sizes to improve readability and maintainability

Replacing magic numbers with named constants can enhance code clarity and make future modifications easier.

Apply this diff to introduce constants:

+ private static final int INTERVALS_PER_DAY_96 = 96;
+ private static final int INTERVALS_PER_DAY_24 = 24;

  ...

  if (qtyByPositionMap.size() == INTERVALS_PER_DAY_96) {
      // existing logic
  } else if (qtyByPositionMap.size() == INTERVALS_PER_DAY_24) {
      // existing logic
  } else {
      // existing logic
  }
cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_CH-IT1-test.xml (1)

Line range hint 22-24: Correct the TimeInterval to match the number of intervals defined

The <TimeInterval> is set to a 2-hour period, but there are 24 intervals defined with a resolution of 1 hour. This inconsistency can lead to incorrect mapping of intervals to time slots.

Adjust the TimeInterval to cover the full range of intervals:

- <TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
+ <TimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />
cse-cc-import-runner-app/src/test/java/com/farao_community/farao/cse/import_runner/app/services/FileImporterTest.java (1)

201-201: Avoid asserting on exact exception messages to reduce brittleness

Asserting exact exception messages can make tests fragile. If the message changes, even slightly, the test will fail. Consider asserting on the exception type or checking that the message contains key information.

Update the assertion to check for relevant content:

- assertEquals("CapacityTimeSeries contains 7 intervals which is different to 24 or 96", nestedE.getMessage());
+ assertTrue(nestedE.getMessage().contains("CapacityTimeSeries contains"));
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 1183af8 and 5308ad3.

📒 Files selected for processing (14)
  • cse-cc-import-runner-app/src/main/java/com/farao_community/farao/cse/import_runner/app/util/Ntc2Util.java (3 hunks)
  • cse-cc-import-runner-app/src/test/java/com/farao_community/farao/cse/import_runner/app/services/FileImporterTest.java (3 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test-96.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test-missing-intervals.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_CH-IT1-test-96.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_CH-IT1-test-missing-intervals.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_CH-IT1-test.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test-96.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test-missing-intervals.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_SI-IT1-test-96.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_SI-IT1-test-missing-intervals.xml (1 hunks)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_SI-IT1-test.xml (1 hunks)
✅ Files skipped from review due to trivial changes (6)
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test-96.xml
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test-missing-intervals.xml
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_CH-IT1-test-96.xml
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test-96.xml
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test-missing-intervals.xml
  • cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_CH-IT1-test-missing-intervals.xml
🔇 Additional comments (7)
cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_FR-IT1-test.xml (1)

36-119: Verify the quantity pattern for new intervals

The newly added intervals 4-24 all have the same quantity value of 99, while the first three intervals have varying values (111, 557, 4096). While this might be intentional for testing, please verify:

  • If this pattern is suitable for the test scenario
  • If it provides adequate coverage for edge cases
  • If it aligns with the test objectives mentioned in PR description about "Adaptation to 96TS"
✅ Verification successful

Based on the shell script results, I can now generate a final response as the pattern is clear across multiple test files.

Quantity pattern of 99 is consistent with test design

The search across all similar test files reveals a consistent pattern where:

  • Each test file has unique values for the first 3 intervals (e.g., FR-IT1: 111,557,4096; AT-IT1: 399,488,1024; SI-IT1: 111,669,8192)
  • All subsequent intervals use 99 as a standard test value
  • This pattern is maintained across regular, 96TS (-96 suffix), and missing-intervals test variations

This consistent pattern across all test files indicates it's an intentional test design choice where:

  • The first 3 intervals test unique/edge cases with specific values
  • The remaining intervals use a common value (99) to verify basic interval handling
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for similar test files to compare quantity patterns
fd -e xml -x grep -l "Qty.*99" {} \; | while read -r file; do
    echo "=== $file ==="
    grep -A 1 "<Qty" "$file" | head -n 10
done

Length of output: 8421

cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_SI-IT1-test.xml (1)

36-119: Review test data pattern for realistic scenarios

The newly added intervals (4-24) all use a uniform quantity value of 99, while the original intervals (1-3) have varying quantities (111, 669, 8192). While this might be suitable for basic testing, consider:

  1. Adding test cases with varying quantities to better represent real-world scenarios
  2. Adding edge cases (e.g., zero or negative quantities) to test boundary conditions

Let's verify if there are other test files with similar patterns:

cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_SI-IT1-test-missing-intervals.xml (3)

33-34: Verify the validity of extreme capacity value

The quantity value of 8192 MAW in position 3 appears unusually high compared to other values (mostly 99 MAW). Please verify if this is an expected test case value.

✅ Verification successful

High capacity values are expected in test files

Looking at the search results, the value of 8192 MAW appears consistently in multiple SI-IT1 test files, and other test files show similar power-of-2 values:

  • SI-IT1 test files: 8192 MAW
  • FR-IT1 test files: 4096 MAW
  • CH-IT1 test files: 2048 MAW
  • AT-IT1 test files: 1024 MAW

These consistent power-of-2 values across different test files indicate they are intentionally chosen test values, likely to verify system behavior with different magnitudes of capacity.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for similar quantity patterns in other test files
rg "<Qty v=\"[0-9]{4,}\"" --type xml

Length of output: 10105


22-23: ⚠️ Potential issue

Time interval and data points mismatch needs investigation

The time interval specified ("2021-06-23T23:00Z/2021-06-24T01:00Z") spans 2 hours, but the file contains 18 intervals with hourly resolution (PT60M). This appears to be inconsistent with the expected data structure.

Also applies to: 24-95


37-95: Consider documenting the significance of repeated value

Multiple intervals (positions 4-18) contain the same quantity value (99). If this is intentional test data, consider adding a comment in the test class explaining the significance of this pattern.

cse-cc-import-runner-app/src/main/java/com/farao_community/farao/cse/import_runner/app/util/Ntc2Util.java (1)

13-13: 🛠️ Refactor suggestion

Consider using specific time zones instead of 'CET' to handle daylight saving time accurately

The use of 'CET' as a time zone ID can be ambiguous during daylight saving time transitions. It's recommended to use a specific time zone like 'Europe/Paris' or 'Europe/Berlin' to ensure correct time calculations.

Apply this diff to update the time zone:

- ZonedDateTime targetDateInCETZone = targetDateTime.atZoneSameInstant(ZoneId.of("CET"));
+ ZonedDateTime targetDateInCETZone = targetDateTime.atZoneSameInstant(ZoneId.of("Europe/Paris"));

Likely invalid or redundant comment.

cse-cc-import-runner-app/src/test/java/com/farao_community/farao/cse/import_runner/app/services/FileImporterTest.java (1)

143-143: Verify the correctness of the time zone conversion in the test

The comment indicates that 2021-06-24T00:30Z is 02:30 in CET, but CET is typically UTC+1. During daylight saving time, it becomes CEST (UTC+2). Ensure that the time conversion aligns with the expected time zone.

Run the following script to confirm the time zone conversion:

✅ Verification successful

The time zone conversion in the comment is correct

The code in DateTimeUtil.java confirms that the application consistently uses CET (Central European Time) zone, and the time conversion in the comment is accurate. During June 24, 2021, CET was in daylight saving time (CEST, UTC+2), so 00:30 UTC correctly converts to 02:30 CEST.

This is evidenced by:

  • The codebase consistently uses ZoneId.of("CET") which automatically handles daylight saving time
  • The Ntc2Util class specifically converts target times to CET zone for processing using atZoneSameInstant(ZoneId.of("CET"))
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify time zone conversion between UTC and CET for the given date.

# Expected output: 02:30 if CET is UTC+2 on the given date.
date -d '2021-06-24T00:30Z' '+%Y-%m-%d %H:%M %Z' --utc
date -d '2021-06-24T00:30Z' '+%Y-%m-%d %H:%M %Z' --timezone='Europe/Paris'

Length of output: 240


Script:

#!/bin/bash
# Let's search for any other time zone related tests or configurations in the codebase
rg -l "T00:30Z|CET|CEST|TimeZone" --type java

# Also search for any time zone related constants or configurations
ast-grep --pattern 'ZoneId.$_' --lang java
ast-grep --pattern 'ZoneOffset.$_' --lang java

Length of output: 538


Script:

#!/bin/bash
# Let's examine the content of DateTimeUtil.java which might contain timezone definitions
cat "cse-lib/data/src/main/java/com/farao_community/farao/cse/data/DateTimeUtil.java"

# And check the Ntc2Util for any timezone handling
cat "cse-cc-import-runner-app/src/main/java/com/farao_community/farao/cse/import_runner/app/util/Ntc2Util.java"

Length of output: 7343

Comment on lines +1 to +98
<?xml version="1.0" encoding="utf-8"?>
<CapacityDocument xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="capacity-document.xsd" DtdVersion="4" DtdRelease="0">
<DocumentIdentification v="NTC2_20221202_2D5_AT-IT-test" />
<DocumentVersion v="1" />
<DocumentType v="A31" />
<ProcessType v="A01" />
<SenderIdentification v="testsender" codingScheme="A01" />
<SenderRole v="A04" />
<ReceiverIdentification v="testrecceiver" codingScheme="A01" />
<ReceiverRole v="A04" />
<CreationDateTime v="2022-12-01T06:07:03Z" />
<CapacityTimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />
<Domain v="testdomain" codingScheme="A01" />
<CapacityTimeSeries>
<TimeSeriesIdentification v="20221202-0001" />
<BusinessType v="A27" />
<Product v="8716867000016"/>
<InArea v="10YIT-GRTN-----B" codingScheme="A01"/>
<OutArea v="10YSI-ELES-----O" codingScheme="A01"/>
<MeasureUnit v="MAW" />
<Period>
<TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
<Resolution v="PT60M" />
<Interval>
<Pos v="1" />
<Qty v="111" />
</Interval>
<Interval>
<Pos v="2" />
<Qty v="669" />
</Interval>
<Interval>
<Pos v="3" />
<Qty v="8192" />
</Interval>
<Interval>
<Pos v="4" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="5" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="6" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="7" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="8" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="9" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="10" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="11" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="12" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="13" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="14" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="15" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="16" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="17" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="18" />
<Qty v="99" />
</Interval>
</Period>
</CapacityTimeSeries>
</CapacityDocument>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Filename suggests missing intervals but all intervals are present

The filename contains "missing-intervals" but the XML contains a complete sequence of intervals from 1 to 18. This inconsistency between the filename and content might lead to confusion in test scenarios.

Consider either:

  1. Removing some intervals to match the filename's implication
  2. Renaming the file to better reflect its actual content

Comment on lines +36 to +119
<Interval>
<Pos v="4" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="5" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="6" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="7" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="8" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="9" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="10" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="11" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="12" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="13" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="14" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="15" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="16" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="17" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="18" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="19" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="20" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="21" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="22" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="23" />
<Qty v="99" />
</Interval>
<Interval>
<Pos v="24" />
<Qty v="99" />
</Interval>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Verify NTC values in test files

Based on the analysis of the test files:

  1. The production NTC2 file (NTC2_20210901_2D1_AT-IT1.xml) shows realistic NTC values ranging from 227 to 253 MW, with values being consistent across multiple intervals.

  2. The test file with 96 intervals (NTC2_20210624_2D5_AT-IT1-test-96.xml) shows:

    • First 3 intervals: Real-looking values (399, 488, 1024)
    • Middle intervals: Placeholder value of 99
    • Last intervals: Zero values

The value 99 appears to be a deliberate test placeholder rather than a realistic NTC value, as it's significantly lower than both:

  • The production values (227-253 MW)
  • The initial test intervals (399-1024 MW)
🔗 Analysis chain

Verify the data pattern and schema compliance

Observations about the data pattern:

  1. First 3 intervals have varying NTC values (399, 488, 1024)
  2. All new intervals (4-24) have the same value (99)

Questions to verify:

  1. Is 99 a valid placeholder or should it contain real NTC values?
  2. Does the XML schema allow 24 intervals for this period?

Let's verify the schema constraints and typical NTC values:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify XML schema constraints and analyze NTC patterns

# Check if schema exists and validate document structure
fd -e xsd capacity-document.xsd

# Look for similar NTC2 test files to compare value patterns
fd -e xml -x grep -l "NTC2.*AT-IT" {}

Length of output: 2082


Script:

#!/bin/bash
# Let's analyze the NTC values in similar test files to understand the patterns

# Check NTC values in AT-IT1 test files
echo "=== AT-IT1 NTC values ==="
cat "./cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210901_2D1_AT-IT1.xml" | grep -A1 "<Qty"

# Check NTC values in test files with -96 suffix which might indicate complete 24-hour data
echo -e "\n=== Complete 24-hour test file NTC values ==="
cat "./cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test-96.xml" | grep -A1 "<Qty"

# Check if there are any comments or documentation about test data values
echo -e "\n=== Comments about test data ==="
rg -i "test.*ntc|ntc.*test" -A 2 -B 2

Length of output: 60976


Script:

#!/bin/bash
# Let's analyze the real NTC values in production files and compare with test files

# Check NTC values in production file
echo "=== Production NTC2 file values ==="
cat "./cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210901_2D1_AT-IT1.xml" | grep -A1 "<Qty"

# Check if there are any test files with 96 intervals to understand the expected pattern
echo -e "\n=== Test file with 96 intervals ==="
cat "./cse-cc-import-runner-app/src/test/resources/com/farao_community/farao/cse/import_runner/app/services/NTC2_20210624_2D5_AT-IT1-test-96.xml" | grep -A1 "<Qty"

# Look for any documentation about test data values in source code
echo -e "\n=== Documentation about test data ==="
rg -i "test.*data|mock.*data" -A 2 -B 2

Length of output: 30439

throw new CseDataException(String.format("CapacityTimeSeries contains %s intervals which is different to 24 or 96", qtyByPositionMap.size()));
}

return qtyByPositionMap.get(position);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Handle the possibility of a null value when retrieving from the map

The method qtyByPositionMap.get(position) may return null if the position key does not exist, potentially leading to a NullPointerException.

Consider adding a null check:

- return qtyByPositionMap.get(position);
+ Double quantity = qtyByPositionMap.get(position);
+ if (quantity == null) {
+     throw new CseDataException("No quantity found for position: " + position);
+ }
+ return quantity;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
return qtyByPositionMap.get(position);
Double quantity = qtyByPositionMap.get(position);
if (quantity == null) {
throw new CseDataException("No quantity found for position: " + position);
}
return quantity;

Comment on lines 52 to 58
if (qtyByPositionMap.size() == 96) {
position = 1 + (4 * targetDateInCETZone.getHour()) + (targetDateInCETZone.getMinute() / 15);
} else if (qtyByPositionMap.size() == 24) {
position = targetDateInCETZone.getHour() + 1;
} else {
throw new CseDataException(String.format("CapacityTimeSeries contains %s intervals which is different to 24 or 96", qtyByPositionMap.size()));
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add validation for minutes to ensure they are multiples of 15

When calculating the position for 96 intervals, the minutes should be multiples of 15 to align with 15-minute intervals. If targetDateInCETZone.getMinute() is not a multiple of 15, it could result in incorrect position calculations.

Consider adding a validation check:

+ if (qtyByPositionMap.size() == 96) {
+     if (targetDateInCETZone.getMinute() % 15 != 0) {
+         throw new CseDataException("Minutes must be a multiple of 15 for 96 intervals");
+     }
      position = 1 + (4 * targetDateInCETZone.getHour()) + (targetDateInCETZone.getMinute() / 15);
  } else if (qtyByPositionMap.size() == 24) {
      position = targetDateInCETZone.getHour() + 1;

Committable suggestion skipped: line range outside the PR's diff.

Comment on lines +22 to +23
<TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
<Resolution v="PT60M" />
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Update TimeInterval and Resolution to align with 96 intervals

The current <TimeInterval> spans 2 hours with a <Resolution> of 60 minutes, but 96 intervals are defined. For 96 intervals over 24 hours, the resolution should be 15 minutes, and the TimeInterval should cover a full day.

Modify the TimeInterval and Resolution as follows:

- <TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
- <Resolution v="PT60M" />
+ <TimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />
+ <Resolution v="PT15M" />
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
<TimeInterval v="2021-06-23T23:00Z/2021-06-24T01:00Z" />
<Resolution v="PT60M" />
<TimeInterval v="2021-06-23T23:00Z/2021-06-24T23:00Z" />
<Resolution v="PT15M" />

@kahyami kahyami requested a review from danielthirion December 3, 2024 13:00
@danielthirion danielthirion merged commit da4daf0 into master Dec 4, 2024
4 checks passed
@danielthirion danielthirion deleted the adaptation_ntc2_to_96_ts branch December 4, 2024 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants