Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLOUD-2261] handling txn recovery by querying api #230

Merged
merged 3 commits into from
Jul 23, 2018
Merged

[CLOUD-2261] handling txn recovery by querying api #230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
87d619e
Updating gitignore for eclipse profile file
ochaloup Jun 29, 2018
6cadec9
[CLOUD-2261] Adding os-partition-txnrecovery module
ochaloup Jun 29, 2018
b918476
[CLOUD-2261] Integrating the os-parition-txnrecovery module for eap s…
ochaloup Jul 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
.idea/
.project

1 change: 1 addition & 0 deletions os-datavirt/module.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ description: Legacy os-datavirt script package.
modules:
install:
- name: os-eap-launch
- name: os-partition
- name: os-eap-migration

execute:
Expand Down
22 changes: 18 additions & 4 deletions os-eap-migration/added/launch/openshift-migrate-common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@ source /opt/partition/partitionPV.sh

function runMigration() {
local instanceDir=$1

# if count provided the node_name should be constructed
local count=$2
[ "x$count" != "x" ] && export NODE_NAME="${NODE_NAME:-node}-${count}"

export NODE_NAME="${NODE_NAME:-node}-${count}"
cp -f ${STANDALONE_XML_COPY} ${STANDALONE_XML}

source $JBOSS_HOME/bin/launch/configure.sh
Expand All @@ -29,9 +31,11 @@ function runMigration() {
local success=false
local message="Finished, migration pod has been terminated"
${JBOSS_HOME}/bin/readinessProbe.sh
local probeStatus=$?

if [ $? -eq 0 ] ; then
if [ $probeStatus -eq 0 ] ; then
echo "$(date): Server started, checking for transactions"

local startTime=$(date +'%s')
local endTime=$((startTime + ${RECOVERY_TIMEOUT} + 1))

Expand All @@ -43,13 +47,23 @@ function runMigration() {
local recoveryClass="com.arjuna.ats.arjuna.tools.RecoveryMonitor"
recoveryJar=$(find "${JBOSS_HOME}" -name \*.jar | xargs grep -l "${recoveryClass}")
if [ -n "${recoveryJar}" ] ; then
echo "$(date): Executing synchronous recovery scan"
echo "$(date): Executing synchronous recovery scan for a first time"
java -cp "${recoveryJar}" "${recoveryClass}" -host "${recoveryHost}" -port "${recoveryPort}"
echo "$(date): Executing synchronous recovery scan for a second time"
java -cp "${recoveryJar}" "${recoveryClass}" -host "${recoveryHost}" -port "${recoveryPort}"
echo "$(date): Synchronous recovery scans finished for the first and the second time"
fi
fi
fi

# -- checking if the pod log is clean from errors (only if function of the particular name exists, provided by the os-partition module)
if [ $probeStatus -eq 0 ] && [ "$(type -t probePodLogForRecoveryErrors)" = 'function' ]; then
probePodLogForRecoveryErrors
probeStatus=$?
[ $probeStatus -ne 0 ] && echo "The migration container log contains periodic recovery errors, check it for details."
fi

if [ $probeStatus -eq 0 ] ; then
while [ $(date +'%s') -lt $endTime -a ! -f "${terminatingFile}" ] ; do
run_cli_cmd '/subsystem=transactions/log-store=log-store/:probe' > /dev/null 2>&1
local transactions="$(run_cli_cmd 'ls /subsystem=transactions/log-store=log-store/transactions')"
Expand All @@ -66,7 +80,7 @@ function runMigration() {
if [ "${success}" = "true" ] ; then
message="Finished, recovery terminated successfully"
else
message="Finished, Recovery DID NOT complete, check log for details. Recovery will be reattempted."
message="Finished, Recovery DID NOT complete, check log for details. Recovery will be reattempted."
fi
fi

Expand Down
4 changes: 0 additions & 4 deletions os-eap-migration/module.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ name: os-eap-migration
version: '1.0'
description: EAP common migration scripts

modules:
install:
- name: os-partition

envs:
- name: "RECOVERY_TIMEOUT"
example: "360"
Expand Down
2 changes: 1 addition & 1 deletion os-eap-probes/added/readinessProbe.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ LOG=/tmp/readiness-log

COUNT=30
SLEEP=5
DEBUG=false
DEBUG=${SCRIPT_DEBUG:-false}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems out of scope, but innocuous.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this helps quite nice during investigation. The cct_module defines the SCRIPT_DEBUG env variable to cause set -x to be set up. Here we got debug information from the python readiness probe as well when the env variable is set. I used that with success while investigating errors.

PROBE_IMPL=probe.eap.dmr.EapProbe

if [ $# -gt 0 ] ; then
Expand Down
3 changes: 0 additions & 3 deletions os-eap64-launch/added/openshift-launch.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@ source ${JBOSS_HOME}/bin/launch/openshift-common.sh

function runServer() {
local instanceDir=$1
local count=$2

export NODE_NAME="${NODE_NAME:-node}-${count}"

source $JBOSS_HOME/bin/launch/configure.sh

Expand Down
1 change: 1 addition & 0 deletions os-eap64-launch/module.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ modules:
install:
- name: os-eap-launch
- name: os-eap-node-name
- name: os-partition-txnrecovery
- name: os-eap-migration

execute:
Expand Down
5 changes: 1 addition & 4 deletions os-eap7-launch/added/openshift-launch.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,12 @@ source $JBOSS_HOME/bin/launch/logging.sh
# TERM signal handler
function clean_shutdown() {
log_error "*** JBossAS wrapper process ($$) received TERM signal ***"
$JBOSS_HOME/bin/jboss-cli.sh -c ":shutdown(timeout=60)"
$JBOSS_HOME/bin/jboss-cli.sh -c "shutdown --timeout=60"
wait $!
}

function runServer() {
local instanceDir=$1
local count=$2

export NODE_NAME="${NODE_NAME:-node}-${count}"

source $JBOSS_HOME/bin/launch/configure.sh

Expand Down
1 change: 1 addition & 0 deletions os-eap7-launch/module.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ modules:
install:
- name: os-eap-launch
- name: os-eap-node-name
- name: os-partition-txnrecovery
- name: os-eap-migration

execute:
Expand Down
1 change: 1 addition & 0 deletions os-jdg-launch/module.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ description: Legacy os-jdg-launch script package.
modules:
install:
- name: os-eap-launch
- name: os-partition
- name: os-eap-migration

execute:
Expand Down
1 change: 1 addition & 0 deletions os-jdg7-launch/module.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ description: Legacy os-jdg7-launch script package.
modules:
install:
- name: os-eap-launch
- name: os-partition
- name: os-eap-migration

execute:
Expand Down
Loading