Code

Optimise ESF#148

Code

Merged

aspacca merged 59 commits intomainelastic/elastic-serverless-forwarder:mainfrom benchmark_for_expand_and_dumperelastic/elastic-serverless-forwarder:benchmark_for_expand_and_dumperCopy head branch name to clipboardAug 17, 2022

+605-299

Commits

Commits on Jul 28, 2022

• Move expand event lisst from field to storage, refactor json collector decorator

  

  Andrea Spacca

  committed
  de2a95a

  Copy full SHA for de2a95a

Commits on Jul 29, 2022

• introduce json_content_type config

  

  Andrea Spacca

  committed
  d019b37

  Copy full SHA for d019b37

• temporary TestLambdaHandlerSuccessS3SQS fix

  

  Andrea Spacca

  committed
  e4b0cc1

  Copy full SHA for e4b0cc1

Commits on Aug 1, 2022

• Update docs

  

  Andrea Spacca

  committed
  42c57d8

  Copy full SHA for 42c57d8

• rename "json" value to "single" value

  

  Andrea Spacca

  committed
  143c103

  Copy full SHA for 143c103

Commits on Aug 2, 2022

• refactor benchmark

  

  Andrea Spacca

  committed
  34b45ee

  Copy full SHA for 34b45ee

• lint

  

  Andrea Spacca

  committed
  9c64a05

  Copy full SHA for 9c64a05

• use continuing queue for kinesis input

  

  Andrea Spacca

  committed
  e885c6e

  Copy full SHA for e885c6e

• adapt test and fix kinesis input continuation

  

  Andrea Spacca

  committed
  9b83d34

  Copy full SHA for 9b83d34

• fix docs

  

  Andrea Spacca

  committed
  cd480d7

  Copy full SHA for cd480d7

• reduce kinesis trigger batch size, remove ReportBatchItemFailures

  

  Andrea Spacca

  committed
  0523c1e

  Copy full SHA for 0523c1e

• add kinesis-data-stream case for _handle_sqs_event

  

  Andrea Spacca

  committed
  e62cd50

  Copy full SHA for e62cd50

• last_event_expanded_offset will be optional

  

  Andrea Spacca

  committed
  7f63b26

  Copy full SHA for 7f63b26

• last_event_expanded_offset in handler

  

  Andrea Spacca

  committed
  c16e44d

  Copy full SHA for c16e44d

• last_event_expanded_offset in cloudwatch logs

  

  Andrea Spacca

  committed
  fc19a86

  Copy full SHA for fc19a86

• last_event_expanded_offset in kinesis data stream

  

  Andrea Spacca

  committed
  88c7cc4

  Copy full SHA for 88c7cc4

• last_event_expanded_offset in s3 notifcation

  

  Andrea Spacca

  committed
  ce65ca3

  Copy full SHA for ce65ca3

• last_event_expanded_offset in sqs

  

  Andrea Spacca

  committed
  daecf7d

  Copy full SHA for daecf7d

• handle continuation from last_event_expanded_offset in expand_event_list_from_field

  

  Andrea Spacca

  committed
  30db89e

  Copy full SHA for 30db89e

• handle optional expanded_ending_offset in decorator

  

  Andrea Spacca

  committed
  fdf034f

  Copy full SHA for fdf034f

• linting and deprecation

  

  Andrea Spacca

  committed
  d86403a

  Copy full SHA for d86403a

• address @todo in integration tests

  

  Andrea Spacca

  committed
  79fc846

  Copy full SHA for 79fc846

• increase coverage

  

  Andrea Spacca

  committed
  58b5baf

  Copy full SHA for 58b5baf

Commits on Aug 3, 2022

• fix test

  

  Andrea Spacca

  committed
  37cb3f4

  Copy full SHA for 37cb3f4

• add benchnmark for expand and dumper

  

  Andrea Spacca

  committed
  d45e18d

  Copy full SHA for d45e18d

• increase rounds in benchmark

  

  Andrea Spacca

  committed
  032051e

  Copy full SHA for 032051e

• Merge branch 'kinesis_input_continuing_queue' into handle-offset-in-expand_event_list_from_field

  

  Andrea Spacca

  authored
  25b4705

  Copy full SHA for 25b4705

• fix return type in _generate

  

  Andrea Spacca

  committed
  b328427

  Copy full SHA for b328427

• Merge branch 'main' into kinesis_input_continuing_queue

  

  Andrea Spacca

  committed
  bab2963

  Copy full SHA for bab2963

• Merge branch 'kinesis_input_continuing_queue' into handle-offset-in-expand_event_list_from_field

  

  Andrea Spacca

  committed
  360cca5

  Copy full SHA for 360cca5

• Merge branch 'handle-offset-in-expand_event_list_from_field' into benchmark_for_expand_and_dumper

  

  Andrea Spacca

  committed
  eb96e97

  Copy full SHA for eb96e97

• Merge branch 'main' into handle-offset-in-expand_event_list_from_field

  

  Andrea Spacca

  committed
  d3ee24c

  Copy full SHA for d3ee24c

• Merge branch 'handle-offset-in-expand_event_list_from_field' into benchmark_for_expand_and_dumper

  

  Andrea Spacca

  committed
  7c8afe3

  Copy full SHA for 7c8afe3

• Merge branch 'main' into benchmark_for_expand_and_dumper

  

  Andrea Spacca

  committed
  104241e

  Copy full SHA for 104241e

Commits on Aug 5, 2022

• let's pass json_content_type to storage factory and increase inflate chunk size

  

  Andrea Spacca

  committed
  cb5e17d

  Copy full SHA for cb5e17d

Commits on Aug 8, 2022

• linting

  

  Andrea Spacca

  committed
  a3309a3

  Copy full SHA for a3309a3

Commits on Aug 9, 2022

• deepcopy is expansive

  

  Andrea Spacca

  committed
  05c3cfc

  Copy full SHA for 05c3cfc

• add disabled value for json_content_type

  

  Andrea Spacca

  committed
  61be493

  Copy full SHA for 61be493

• json serialiser in es, json most performing dumper and parser elsewhere

  

  Andrea Spacca

  committed
  0830a6f

  Copy full SHA for 0830a6f

• fix json_parser in storage decorator

  

  Andrea Spacca

  committed
  60a05d8

  Copy full SHA for 60a05d8

• events_list = events_list[offset_skew:] pysimdjson parse the list content

  

  Andrea Spacca

  committed
  345d4b1

  Copy full SHA for 345d4b1

• fix tests

  

  Andrea Spacca

  committed
  fe579f8

  Copy full SHA for fe579f8

• silence sensitive logging

  

  Andrea Spacca

  committed
  b65cb67

  Copy full SHA for b65cb67

• increase coverage

  

  Andrea Spacca

  committed
  030e10d

  Copy full SHA for 030e10d

• wider exception in parse and dumper for es

  

  Andrea Spacca

  committed
  d6dfa1c

  Copy full SHA for d6dfa1c

Commits on Aug 11, 2022

• increase coverage, remove pysimdjson

  

  Andrea Spacca

  committed
  e7c4afd

  Copy full SHA for e7c4afd

• always use usjon

  

  Andrea Spacca

  committed
  a97c436

  Copy full SHA for a97c436

• fix tests

  

  Andrea Spacca

  committed
  701a0b6

  Copy full SHA for 701a0b6

• linting

  

  Andrea Spacca

  committed
  02d53ef

  Copy full SHA for 02d53ef

• fix tests

  

  Andrea Spacca

  committed
  4d8ae1e

  Copy full SHA for 4d8ae1e

• changelog

  

  Andrea Spacca

  committed
  99d8c5c

  Copy full SHA for 99d8c5c

• typo in changelog

  

  Andrea Spacca

  committed
  c9e06e5

  Copy full SHA for c9e06e5

• a test file left behind

  

  Andrea Spacca

  committed
  e491665

  Copy full SHA for e491665

• tests take longer

  

  Andrea Spacca

  committed
  ad13910

  Copy full SHA for ad13910

Commits on Aug 16, 2022

• cr fixes

  

  Andrea Spacca

  committed
  3324d8b

  Copy full SHA for 3324d8b

Commits on Aug 17, 2022

• fix #151

  

  Andrea Spacca

  committed
  282fa88

  Copy full SHA for 282fa88

• strip message when checking for emoty event

  

  Andrea Spacca

  committed
  d66a793

  Copy full SHA for d66a793

• remove dumping possible sensitive data in the logs

  

  Andrea Spacca

  committed
  53617fd

  Copy full SHA for 53617fd

• lint

  

  Andrea Spacca

  committed
  188d04a

  Copy full SHA for 188d04a