Skip to content
This repository has been archived by the owner on Nov 9, 2022. It is now read-only.

Fixing Issue #223 Capture modules for any app type #356

Merged
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 34 additions & 14 deletions deploy/lib/server_config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -933,9 +933,7 @@ def capture

# check params
if full_config == nil && config == nil && target_db == nil
raise HelpException.new("capture", "either full-ml-config, ml-config or modules-db is required")
elsif target_db != nil && @properties['ml.app-type'] != 'rest'
raise ExitException.new("This is a #{@properties['ml.app-type']} application; capture modules only works for app-type=rest")
raise HelpException.new("capture", "either full-ml-config, ml-config, app-builder or modules-db is required")
end

# retrieve full setup config from environment
Expand All @@ -947,16 +945,24 @@ def capture
if target_db != nil
tmp_dir = Dir.mktmpdir
logger.debug "using temp dir " + tmp_dir
logger.info "Retrieving source and REST config from #{target_db}..."

save_files_to_fs(target_db, "#{tmp_dir}/src")
if (port != nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice way to solve capturing non-REST modules..

logger.info "Retrieving source and REST config from #{target_db}..."
else
logger.info "Retrieving source from #{target_db}..."
end

# set up the options
FileUtils.cp_r(
"#{tmp_dir}/src/#{@properties['ml.group']}/" + target_db.sub("-modules", "") + "/rest-api/.",
@properties['ml.rest-options.dir']
)
FileUtils.rm_rf("#{tmp_dir}/src/#{@properties['ml.group']}/")
# send the target db, the destination directory, and port to know if REST for ML7+
save_files_to_fs(target_db, "#{tmp_dir}/src", port)

if (port != nil)
# set up the options
FileUtils.cp_r(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hitting issue #298, or something very similar. If you start with mvc app-type, the rest-api dir doesn't exist yet. There is code mentioned in the ticket that you could include here to get the dir created.

You could also argue that you might want to throw an error telling that you are capturing REST modules into a non-REST project. That might even be better, as rewriter settings will be off..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like throwing the error idea, that way the user does not end up with something unexpected. I have implemented this and will be committing momentarily. I will note that it also fixes issue #298 since "capture" will no longer try to save REST configuration if the folder does not exists in the project.

"#{tmp_dir}/src/#{@properties['ml.group']}/" + target_db.sub("-modules", "") + "/rest-api/.",
@properties['ml.rest-options.dir']
)
FileUtils.rm_rf("#{tmp_dir}/src/#{@properties['ml.group']}/")
end

# Make sure REST properties are in accurate format, so you can directly deploy them again..
if (port != nil)
Expand All @@ -980,7 +986,7 @@ def capture

private

def save_files_to_fs(target_db, target_dir)
def save_files_to_fs(target_db, target_dir, port)
# Get the list of URIs. We get them in order because Ruby's Dir.mkdir
# command doesn't have a -p option (create parent).
dirs = execute_query %Q{
Expand Down Expand Up @@ -1026,8 +1032,22 @@ def save_files_to_fs(target_db, target_dir)
# create the directory so that it will exist when we try to save files
Dir.mkdir("#{target_dir}" + uri)
else
r = go("#{@protocol}://#{@hostname}:#{@bootstrap_port}/qconsole/endpoints/view.xqy?dbid=#{db_id}&uri=#{uri}", "get")
File.open("#{target_dir}#{uri}", 'wb') { |file| file.write(r.body) }
# check type, if REST use that endpoint, otherwise use a query
if (port != nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is unnecessary. The port value is not used, and the downloading of files depends on QConsole, which works for any app-type..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I have removed this now.

r = go("#{@protocol}://#{@hostname}:#{@bootstrap_port}/qconsole/endpoints/view.xqy?dbid=#{db_id}&uri=#{uri}", "get")
file_content = r.body
else
# Note : There are limitations to doing it this way for non REST DBs; only text files are captured (xqy, html, etc..)
# other types such as images or java scripts will not be grabbed. Also this method is slower than REST or XCC.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is why @dmcassel moved away from this method in #163. And using QConsole approach of above also bypasses this problem I think. I am pretty sure the capture was capturing images of app-builder apps correctly..

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem in #163 was that the REST API (at least at the time) was returning a JSON description of the binary files, instead of the file itself.

r = execute_query %Q{
fn:doc("#{uri}")
},
{ :db_name => target_db }
response_hash = JSON.parse("#{r.body}").each do |r|
file_content = r['result']
end
end
File.open("#{target_dir}#{uri}", 'wb') { |file| file.write(file_content) }
end
end
end
Expand Down