Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update namespace to match the one used by aws-sdk-v1 gem #15

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ executable. A record processor in Ruby typically looks something like:

require 'aws/kclrb'

class SampleRecordProcessor < Aws::KCLrb::RecordProcessorBase
class SampleRecordProcessor < AWS::KCLrb::RecordProcessorBase
def init_processor(shard_id)
# initialize
end
Expand All @@ -32,7 +32,7 @@ executable. A record processor in Ruby typically looks something like:
if __FILE__ == $0
# Start the main processing loop
record_processor = SampleRecordProcessor.new
driver = Aws::KCLrb::KCLProcess.new(record_processor)
driver = AWS::KCLrb::KCLProcess.new(record_processor)
driver.run
end
```
Expand Down Expand Up @@ -85,11 +85,11 @@ To run the data producer, run the following commands:

#### Notes

* The [AWS Ruby SDK gem][aws-ruby-sdk-gem] needs to be installed as a pre-requisite. To install,
* The [AWS Ruby SDK V1 gem][aws-ruby-sdk-v1-gem] needs to be installed as a pre-requisite. To install,
run:

```sh
sudo gem install aws-sdk
sudo gem install aws-sdk-v1
```

* The script `samples/sample_kcl_producer.rb` takes several parameters that you can use
Expand Down
2 changes: 1 addition & 1 deletion lib/aws/kclrb/checkpointer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

require 'aws/kclrb/io_proxy'

module Aws
module AWS
module KCLrb
# Error class used for wrapping exception names passed through the
# input stream.
Expand Down
4 changes: 2 additions & 2 deletions lib/aws/kclrb/io_proxy.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

require 'multi_json'

module Aws
module AWS
module KCLrb
# @api private
# Internal class used by {KCLProcess} and {Checkpointer} to communicate
Expand Down Expand Up @@ -96,4 +96,4 @@ def write_action(action, details={})
end
end
end
end
end
2 changes: 1 addition & 1 deletion lib/aws/kclrb/kcl_process.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
require 'aws/kclrb/io_proxy'
require 'aws/kclrb/checkpointer'

module Aws
module AWS
module KCLrb
# Error raised if the {KCLProcess} received an input action that it
# could not parse or it could not handle.
Expand Down
4 changes: 2 additions & 2 deletions lib/aws/kclrb/record_processor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.

module Aws
module AWS
module KCLrb
# @abstract
# Base class for implementing a record processor.
Expand Down Expand Up @@ -74,4 +74,4 @@ def shutdown(checkpointer, reason)
end
end
end
end
end
18 changes: 9 additions & 9 deletions samples/sample_kcl.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@
require 'fileutils'

# @api private
# A sample implementation of the {Aws::KCLrb::RecordProcessorBase RecordProcessor}.
# A sample implementation of the {AWS::KCLrb::RecordProcessorBase RecordProcessor}.
#
# All it does is write the data to an output stream. Be careful not to use
# the `$stdout` as it's used to communicate with the {https://github.com/awslabs/amazon-kinesis-client/blob/master/src/main/java/com/amazonaws/services/kinesis/multilang/package-info.java MultiLangDaemon}.
# If you use `$stderr` instead the MultiLangDaemon would echo the output
# to its own standard error stream.
class SampleRecordProcessor < Aws::KCLrb::RecordProcessorBase
class SampleRecordProcessor < AWS::KCLrb::RecordProcessorBase
# @param output [IO, String] If a string is provided, it's assumed to be the path
# to an output directory. That directory would be created and permissions to write
# to it are asserted.
Expand All @@ -48,7 +48,7 @@ def initialize(output=$stderr)
end
end

# (see Aws::KCLrb::RecordProcessorBase#init_processor)
# (see AWS::KCLrb::RecordProcessorBase#init_processor)
def init_processor(shard_id)
unless @output
@filename = File.join(@output_directory, "#{shard_id}-#{Time.now.to_i}.log")
Expand All @@ -57,7 +57,7 @@ def init_processor(shard_id)
end
end

# (see Aws::KCLrb::RecordProcessorBase#process_records)
# (see AWS::KCLrb::RecordProcessorBase#process_records)
def process_records(records, checkpointer)
last_seq = nil
records.each do |record|
Expand All @@ -74,7 +74,7 @@ def process_records(records, checkpointer)
checkpoint_helper(checkpointer, last_seq) if last_seq
end

# (see Aws::KCLrb::RecordProcessorBase#shutdown)
# (see AWS::KCLrb::RecordProcessorBase#shutdown)
def shutdown(checkpointer, reason)
checkpoint_helper(checkpointer) if 'TERMINATE' == reason
ensure
Expand All @@ -84,12 +84,12 @@ def shutdown(checkpointer, reason)

private
# Helper method that retries checkpointing once.
# @param checkpointer [Aws::KCLrb::Checkpointer] The checkpointer instance to use.
# @param sequence_number (see Aws::KCLrb::Checkpointer#checkpoint)
# @param checkpointer [AWS::KCLrb::Checkpointer] The checkpointer instance to use.
# @param sequence_number (see AWS::KCLrb::Checkpointer#checkpoint)
def checkpoint_helper(checkpointer, sequence_number=nil)
begin
checkpointer.checkpoint(sequence_number)
rescue Aws::KCLrb::CheckpointError => e
rescue AWS::KCLrb::CheckpointError => e
# Here, we simply retry once.
# More sophisticated retry logic is recommended.
checkpointer.checkpoint(sequence_number) if sequence_number
Expand All @@ -100,7 +100,7 @@ def checkpoint_helper(checkpointer, sequence_number=nil)
if __FILE__ == $0
# Start the main processing loop
record_processor = SampleRecordProcessor.new(ARGV[1] || File.join(Dir.tmpdir, 'kclrbsample'))
driver = Aws::KCLrb::KCLProcess.new(record_processor)
driver = AWS::KCLrb::KCLProcess.new(record_processor)
driver.run
end

2 changes: 1 addition & 1 deletion spec/checkpointer_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
require 'aws/kclrb/io_proxy.rb'
require 'aws/kclrb/checkpointer.rb'

module Aws::KCLrb
module AWS::KCLrb
describe Checkpointer do
describe "#checkpoint" do
it "should emit a checkpoint action and consume response action" do
Expand Down
2 changes: 1 addition & 1 deletion spec/io_proxy_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

require 'aws/kclrb/io_proxy'

module Aws::KCLrb
module AWS::KCLrb
describe IOProxy do
describe "#read_line" do
it "should skip blank lines" do
Expand Down
2 changes: 1 addition & 1 deletion spec/kcl_process_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
require 'aws/kclrb/kcl_process'
require 'aws/kclrb/record_processor'

module Aws::KCLrb
module AWS::KCLrb
# Dummy test class.
# The {#process_reocrds} method will retry the checkpointing call
# in case of a throttling exception.
Expand Down