-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement the SAP systems payload decoding and usage #602
Merged
Merged
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
349 changes: 349 additions & 0 deletions
349
lib/trento/application/integration/discovery/payloads/sap_system_discovery_payload.ex
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,349 @@ | ||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload do | ||
@moduledoc """ | ||
SAP system discovery integration event payload | ||
""" | ||
|
||
alias Trento.Integration.Discovery.SapSystemDiscoveryPayload.{ | ||
Database, | ||
Instance, | ||
Profile | ||
} | ||
|
||
@required_fields [:Id, :SID, :Type, :Profile, :Databases, :Instances] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
field :Id, :string | ||
field :SID, :string | ||
field :Type, :integer | ||
field :DBAddress, :string | ||
|
||
embeds_one :Profile, Profile | ||
embeds_many :Databases, Database | ||
embeds_many :Instances, Instance | ||
end | ||
|
||
def changeset(sap_system, attrs) do | ||
modified_attrs = | ||
attrs | ||
|> databases_to_list | ||
|
||
sap_system | ||
|> cast(modified_attrs, fields()) | ||
|> cast_embed(:Profile, with: {Profile, :changeset, [parse_system_type(attrs)]}) | ||
|> cast_embed(:Databases) | ||
|> cast_embed(:Instances) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
|
||
defp parse_system_type(%{"Type" => system_type}), do: system_type | ||
defp parse_system_type(_), do: nil | ||
|
||
defp databases_to_list(%{"Databases" => nil} = attrs), | ||
do: %{attrs | "Databases" => []} | ||
|
||
defp databases_to_list(attrs), do: attrs | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.Profile do | ||
@moduledoc """ | ||
Profile field payload | ||
""" | ||
|
||
@application_type 2 | ||
@application_required_fields [:"dbs/hdb/dbname"] | ||
|
||
# Cannot use Trento.Type here, Jason.Encoder is breaking the schema creation | ||
use Ecto.Schema | ||
import Ecto.Changeset | ||
|
||
@type t() :: %__MODULE__{} | ||
|
||
@primary_key false | ||
|
||
embedded_schema do | ||
field :"dbs/hdb/dbname", :string | ||
end | ||
|
||
def changeset(profile, attrs, type) do | ||
profile | ||
|> cast(attrs, __MODULE__.__schema__(:fields)) | ||
|> maybe_validate_required_fields(type) | ||
end | ||
|
||
defp maybe_validate_required_fields(changeset, @application_type), | ||
do: | ||
changeset | ||
|> validate_required(@application_required_fields) | ||
|
||
defp maybe_validate_required_fields(changeset, _), do: changeset | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.Database do | ||
@moduledoc """ | ||
Databases field payload | ||
""" | ||
|
||
@required_fields [:Database] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
field :Host, :string | ||
field :User, :string | ||
field :Group, :string | ||
field :Active, :string | ||
field :UserId, :string | ||
field :GroupId, :string | ||
field :SqlPort, :string | ||
field :Database, :string | ||
field :Container, :string | ||
end | ||
|
||
def changeset(database, attrs) do | ||
database | ||
|> cast(attrs, fields()) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.Instance do | ||
@moduledoc """ | ||
Instances field payload | ||
""" | ||
|
||
alias Trento.Integration.Discovery.SapSystemDiscoveryPayload.{ | ||
SapControl, | ||
SystemReplication | ||
} | ||
|
||
@required_fields [:Host, :Name, :Type, :SAPControl, :SystemReplication] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
field :Host, :string | ||
field :Name, :string | ||
field :Type, :integer | ||
|
||
embeds_one :SAPControl, SapControl | ||
embeds_one :SystemReplication, SystemReplication | ||
end | ||
|
||
def changeset(instance, attrs) do | ||
instance | ||
|> cast(attrs, fields()) | ||
|> cast_embed(:SAPControl) | ||
|> cast_embed(:SystemReplication) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.SapControl do | ||
@moduledoc """ | ||
SAP control field payload | ||
""" | ||
|
||
alias Trento.Integration.Discovery.SapSystemDiscoveryPayload.{ | ||
SapControlInstance, | ||
SapControlProcess, | ||
SapControlProperty | ||
} | ||
|
||
@required_fields [:Properties, :Instances, :Processes] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
embeds_many :Properties, SapControlProperty | ||
embeds_many :Instances, SapControlInstance | ||
embeds_many :Processes, SapControlProcess | ||
end | ||
|
||
def changeset(sap_control, attrs) do | ||
hostname = find_property("SAPLOCALHOST", attrs) | ||
|
||
instance_number = | ||
case find_property("SAPSYSTEM", attrs) do | ||
instance_number when is_binary(instance_number) -> String.to_integer(instance_number) | ||
_ -> nil | ||
end | ||
|
||
sap_control | ||
|> cast(attrs, fields()) | ||
|> cast_embed(:Properties) | ||
|> cast_embed(:Instances, with: {SapControlInstance, :changeset, [hostname, instance_number]}) | ||
|> cast_embed(:Processes) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
|
||
def find_property(property, %{"Properties" => properties}) do | ||
properties | ||
|> Enum.find_value(fn | ||
%{"property" => ^property, "value" => value} -> value | ||
_ -> nil | ||
end) | ||
end | ||
|
||
def find_property(_, _), do: nil | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.SapControlProperty do | ||
@moduledoc """ | ||
SAP control property field payload | ||
""" | ||
|
||
@required_fields [:value, :property] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
field :value, :string | ||
field :property, :string | ||
field :propertytype, :string | ||
end | ||
|
||
def changeset(property, attrs) do | ||
property | ||
|> cast(attrs, fields()) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.SapControlInstance do | ||
@moduledoc """ | ||
SAP control instances field payload | ||
""" | ||
|
||
@required_fields [ | ||
:features, | ||
:hostname, | ||
:httpPort, | ||
:httpsPort, | ||
:dispstatus, | ||
:instanceNr, | ||
:startPriority | ||
] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
# current_instance is a custom field to make data extraction easier | ||
field :current_instance, :boolean, default: false | ||
field :features, :string | ||
field :hostname, :string | ||
field :httpPort, :integer | ||
field :httpsPort, :integer | ||
|
||
field :dispstatus, Ecto.Enum, | ||
values: [ | ||
:"SAPControl-GREEN", | ||
:"SAPControl-YELLOW", | ||
:"SAPControl-RED", | ||
:"SAPControl-GRAY" | ||
] | ||
|
||
field :instanceNr, :integer | ||
field :startPriority, :string | ||
end | ||
|
||
def changeset(instance, attrs, hostname, instance_number) do | ||
enriched_attrs = enrich_current_instance(attrs, hostname, instance_number) | ||
|
||
instance | ||
|> cast(enriched_attrs, fields()) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
|
||
defp enrich_current_instance( | ||
%{"hostname" => current_hostname, "instanceNr" => current_instance_number} = attrs, | ||
hostname, | ||
instance_number | ||
) | ||
when hostname == current_hostname and instance_number == current_instance_number, | ||
do: Map.put(attrs, "current_instance", true) | ||
|
||
defp enrich_current_instance(attrs, _, _), do: attrs | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.SapControlProcess do | ||
@moduledoc """ | ||
SAP control process field payload | ||
""" | ||
|
||
@required_fields [ | ||
:pid, | ||
:name, | ||
:starttime, | ||
:dispstatus, | ||
:textstatus, | ||
:description, | ||
:elapsedtime | ||
] | ||
|
||
use Trento.Type | ||
|
||
deftype do | ||
field :pid, :integer | ||
field :name, :string | ||
field :starttime, :string | ||
|
||
field :dispstatus, Ecto.Enum, | ||
values: [ | ||
:"SAPControl-GREEN", | ||
:"SAPControl-YELLOW", | ||
:"SAPControl-RED", | ||
:"SAPControl-GRAY" | ||
] | ||
|
||
field :description, :string | ||
field :elapsedtime, :string | ||
end | ||
|
||
def changeset(process, attrs) do | ||
process | ||
|> cast(attrs, fields()) | ||
|> validate_required_fields(@required_fields) | ||
end | ||
end | ||
|
||
defmodule Trento.Integration.Discovery.SapSystemDiscoveryPayload.SystemReplication do | ||
arbulu89 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
@moduledoc """ | ||
SystemReplication process field payload | ||
""" | ||
|
||
@required_fields [:local_site_id] | ||
|
||
# Cannot use Trento.Type here, Jason.Encoder is breaking the schema creation | ||
use Ecto.Schema | ||
import Ecto.Changeset | ||
|
||
@type t() :: %__MODULE__{} | ||
|
||
@primary_key false | ||
|
||
embedded_schema do | ||
field :local_site_id, :string | ||
field :overall_replication_status, :string | ||
field :"site/1/REPLICATION_MODE", :string | ||
field :"site/2/REPLICATION_MODE", :string | ||
end | ||
|
||
def changeset(system_replication, attrs) do | ||
local_site_id = parse_local_site_id(attrs) | ||
|
||
system_replication | ||
|> cast(attrs, __MODULE__.__schema__(:fields)) | ||
|> validate_required(@required_fields) | ||
|> validate_replication_mode(local_site_id) | ||
end | ||
|
||
def parse_local_site_id(%{"local_site_id" => local_site_id}), do: local_site_id | ||
def parse_local_site_id(_), do: 1 | ||
|
||
defp validate_replication_mode(changeset, local_site_id) do | ||
changeset | ||
|> validate_required([:"site/#{local_site_id}/REPLICATION_MODE"]) | ||
end | ||
end |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAIK, fields should be starting with a lowercase.
Is this related to your comment
I couldn't use the ProperCase option to snake case all the keys, as it creates some really strange entries ?
If so I couldn't fully understand what is happening 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fields are exactly what we receive in the discovery json payload. In this case we receive the
Id
, as it is, starting upper case.In other piece of code, @dottorblaster used the
ProperCase.snake_case
function to convert the coming payload to snake-cased payload, so he could have snake case fields. In essence, the payload schemas he created doesn't 100% match with the coming payload, as he is pre-snake-casing (nice word XD).The reason why I didn't use the
ProperCase.snake_case
function is that it creates really strange keys, because our coming payload has strange keys indeed. Many of them because we simply send but SAP commands give us...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arbulu89
could you please make an example of what are these strange keys? maybe I would be able to help with this or even rewrite our version of
ProperCase
that fits for our use-case (including the "/" keys)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fabriziosestito
Here some few:
HdbnsutilSRstate
tohdbnsutil_s_rstate
icm/HTTP/ASJava/disable_url_session_tracking
toicm/_htt_p/_as_java/disable_url_session_tracking
is/HTTP/show_detailed_errors
tois/_htt_p/show_detailed_errors
siteTier/NBG
tosite_tier/_nbg
From this list, we really only use the 1st one. I didn't want to start having this strange fields in the code.
I thought of changing the agent side too, but I didn't want to start on this until having the payload versioning system on place. In in may cases, these strings are coming directly from SAP utilities, so we cannot do changes that easy.
If you don't mind having these fields, we can snake case anyway
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arbulu89
I see thanks. I started playing around but I ended up with the same problem as the propercase library. It uses Macro.underscore under the hood which causes the "weird" casing. We will want to change the agent side in the future I agree. Since there is no easy way to do this, and what you proposed is limited to the payload decoded (and not to the commands itself) I would think this is good enough (even if I cringe everytime I see a camlecase atom field 😬 )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can have poll hehe;
Who prefers the:
Either way, there will be strange fields.