-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kirill Hryvicki - 0 #20
base: master
Are you sure you want to change the base?
Changes from 2 commits
3795389
c595793
ee82336
2f58ca2
f282d32
f427a7e
b3dcf46
56858bd
98ab240
e7812bc
aae184e
a497d5f
bfa5dcb
a970933
1dd7ba8
448c419
ad5d76d
c477f85
2d51305
8fecc37
5c964c4
38add5b
8097555
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
require 'mechanize' | ||
require 'open-uri' | ||
require 'uri' | ||
|
||
agent = Mechanize.new | ||
page = agent.get('http://www.belstat.gov.by/ofitsialnaya-statistika/makroekonomika-i-okruzhayushchaya-sreda/tseny/operativnaya-informatsiya_4/srednie-tseny-na-potrebitelskie-tovary-i-uslugi-po-respublike-belarus') | ||
page.links_with(:href => /.xls/).each do |link| | ||
str_link = link.href.to_s | ||
str_link = URI.unescape(str_link) | ||
str_link = URI.escape(str_link) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lint/UriEscapeUnescape: URI.escape method is obsolete and should not be used. Instead, use CGI.escape, URI.encode_www_form or URI.encode_www_form_component depending on your specific use case. |
||
if /http:\/\/www.belstat.gov.by/ === str_link | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/ConditionalAssignment: Use the return of the conditional for variable assignment and comparison. |
||
f_link = str_link | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/IndentationWidth: Use 2 (not 4) spaces for indentation. |
||
else | ||
f_link = 'http://www.belstat.gov.by' + str_link | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/IndentationWidth: Use 2 (not 4) spaces for indentation. |
||
end | ||
download = open(f_link) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Security/Open: The use of Kernel#open is a serious security risk. |
||
IO.copy_stream(download, "./data/#{download.base_uri.to_s.split('/')[-1]}") | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/TrailingBlankLines: Final newline missing. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,241 @@ | ||
require 'roo' | ||
RGBD marked this conversation as resolved.
Show resolved
Hide resolved
|
||
require 'roo-xls' | ||
|
||
def getFileInstance(filePath) | ||
ext = filePath.split(".")[2] | ||
fileInstance = nil | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/TrailingWhitespace: Trailing whitespace detected. |
||
if ext == "xls" || ext == "xlsx" | ||
fileInstance = Roo::Spreadsheet.open(filePath) | ||
case ext | ||
when "xls" | ||
fileInstance = Roo::Excel.new(filePath) | ||
when "xlsx" | ||
fileInstance = Roo::Excelx.new(filePath) | ||
end | ||
else | ||
puts "unsupported file type: " + filePath | ||
end | ||
|
||
return fileInstance | ||
end | ||
|
||
def findKeys(word, products) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Naming/MethodName: Use snake_case for method names. |
||
result = [] | ||
|
||
products.keys.each { |key| | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/BlockDelimiters: Avoid using {...} for multi-line blocks. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. можно использовать select, и обойтись без объявления result There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Сделано |
||
if(key.include?(word)) | ||
result.push(key) | ||
end | ||
} | ||
return result | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/RedundantReturn: Redundant return detected. |
||
end | ||
|
||
def fetchProductsData(fileInstance, products, regions) | ||
|
||
for n in 9..fileInstance.last_row | ||
if fileInstance.cell("E", n) == nil | ||
next | ||
end | ||
|
||
year = fileInstance.cell("A", 3).split(" ")[2] | ||
month = fileInstance.cell("A", 3).split(" ")[1] | ||
|
||
key = fileInstance.cell("A", n).strip.downcase | ||
|
||
if !products[key] | ||
products[key] = Hash.new | ||
end | ||
if !products[key][year] | ||
products[key][year] = Hash.new | ||
end | ||
|
||
products[key][year][month] = { | ||
regions[0] => formatValue(fileInstance.cell("G", n), year), | ||
regions[1] => formatValue(fileInstance.cell("I", n), year), | ||
regions[2] => formatValue(fileInstance.cell("K", n), year), | ||
regions[3] => formatValue(fileInstance.cell("M", n), year), | ||
regions[4] => formatValue(fileInstance.cell("O", n), year), | ||
regions[5] => formatValue(fileInstance.cell("Q", n), year), | ||
regions[6] => formatValue(fileInstance.cell("S", n), year) | ||
} | ||
end | ||
end | ||
|
||
def formatValue(val, year) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Naming/MethodName: Use snake_case for method names. |
||
if val | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/GuardClause: Use a guard clause instead of wrapping the code inside a conditional expression. |
||
result = val.to_f | ||
|
||
if year.to_i < 2017 | ||
result = result / 10000 | ||
end | ||
|
||
return result.round(2) | ||
end | ||
end | ||
|
||
def getRecentPriceData(key, products, monthMap) | ||
currentYear = Time.now.strftime("%Y").to_s | ||
|
||
currentMonth = Time.now.strftime("%m") | ||
|
||
monthMap.each { |month, monthNumber| | ||
if monthNumber == currentMonth | ||
currentMonth == month | ||
end | ||
} | ||
|
||
productYearData = products[key]; | ||
|
||
if productYearData[currentYear] | ||
yearKey = currentYear | ||
else | ||
begin | ||
yearKey = productYearData.keys.max{ |a,b| a.to_i <=> b.to_i} | ||
rescue | ||
puts productYearData.keys | ||
end | ||
|
||
end | ||
|
||
productMonthData = productYearData[yearKey] | ||
|
||
if productMonthData[currentMonth] | ||
monthKey = currentMonth | ||
else | ||
monthKey = productMonthData.keys.max{ |a,b| monthMap[a].to_i <=> monthMap[b].to_i} | ||
end | ||
|
||
return { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/RedundantReturn: Redundant return detected. |
||
"price" => productMonthData[monthKey]["Minsk"], | ||
"year" => yearKey, | ||
"month" => monthKey, | ||
"product" => key | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/TrailingWhitespace: Trailing whitespace detected. |
||
end | ||
|
||
def getMinPrice(hash) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Metrics/MethodLength: Method has too many lines. [19/10] |
||
result = Hash.new | ||
minYearPrice = 9999999999999 | ||
|
||
hash.each { |year, yearHash| | ||
minMonthPrice = 9999999999999 | ||
yearHash.each { |month, monthHash| | ||
price = monthHash['Minsk'] | ||
if !price | ||
next | ||
end | ||
if(price < minMonthPrice) | ||
minMonthPrice = price | ||
result['month'] = month | ||
end | ||
} | ||
if(minMonthPrice < minYearPrice) | ||
minYearPrice = minMonthPrice | ||
result['year'] = year | ||
result['price'] = minYearPrice | ||
end | ||
} | ||
return result | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/RedundantReturn: Redundant return detected. |
||
end | ||
|
||
def getMaxPrice(hash) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Metrics/MethodLength: Method has too many lines. [19/10] |
||
result = Hash.new | ||
maxYearPrice = 0 | ||
|
||
hash.each { |year, yearHash| | ||
maxMonthPrice = 0 | ||
yearHash.each { |month, monthHash| | ||
price = monthHash['Minsk'] | ||
if !price | ||
next | ||
end | ||
if(price > maxMonthPrice) | ||
maxMonthPrice = price | ||
result['month'] = month | ||
end | ||
} | ||
if(maxMonthPrice > maxYearPrice) | ||
maxYearPrice = maxMonthPrice | ||
result['year'] = year | ||
result['price'] = maxYearPrice | ||
end | ||
} | ||
return result | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/RedundantReturn: Redundant return detected. |
||
end | ||
|
||
def getSimilarPriceProducts(data, products) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Metrics/AbcSize: Assignment Branch Condition size for getSimilarPriceProducts is too high. [17.23/15] |
||
result = [] | ||
price = data["price"] | ||
year = data["year"] | ||
month = data["month"] | ||
originProduct = data["product"] | ||
|
||
products.each { |product, productData| | ||
if !productData[year] | ||
next | ||
end | ||
if !productData[year][month] | ||
next | ||
end | ||
if productData[year][month]["Minsk"] == price && product != originProduct | ||
result.push(product) | ||
end | ||
} | ||
return result | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/RedundantReturn: Redundant return detected. |
||
end | ||
|
||
def main | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Metrics/AbcSize: Assignment Branch Condition size for main is too high. [61.07/15] There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Metrics/AbcSize: Assignment Branch Condition size for main is too high. [60.09/15] |
||
monthMap = { | ||
'январь' => 1, | ||
'февраль' => 2, | ||
'март' => 3, | ||
'апрель' => 4, | ||
'май' => 5, | ||
'июнь' => 6, | ||
'июль' => 7, | ||
'авуст' => 8, | ||
'сентябрь' => 9, | ||
'октябрь' => 10, | ||
'ноябрь' => 11, | ||
'декабрь' => 12, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/TrailingCommaInHashLiteral: Avoid comma after the last item of a hash. |
||
} | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/TrailingWhitespace: Trailing whitespace detected. |
||
regions = ['Brest', 'Vitebsk', 'Gomel', 'Grodno', 'Minsk', 'Minsk Region', 'Mogilyov'] | ||
products = Hash.new | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/EmptyLiteral: Use hash literal {} instead of Hash.new. |
||
filePaths = Dir["./data/*"] | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/TrailingWhitespace: Trailing whitespace detected. |
||
filePaths.each { |filePath| | ||
fileInstance = getFileInstance(filePath) | ||
if fileInstance | ||
fetchProductsData(fileInstance, products, regions) | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/TrailingWhitespace: Trailing whitespace detected. |
||
} | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/EmptyLines: Extra blank line detected. |
||
while true | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/InfiniteLoop: Use Kernel#loop for infinite loops. |
||
puts "What price are you looking for?" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
word = gets.chomp.downcase | ||
keys = findKeys(word, products) | ||
if keys.length == 0 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/NumericPredicate: Use keys.length.zero? instead of keys.length == 0. |
||
puts word.capitalize + " can not be found in database" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
else | ||
keys.each { |key| | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/BlockDelimiters: Avoid using {...} for multi-line blocks. |
||
recentPriceData = getRecentPriceData(key, products, monthMap) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/IndentationWidth: Use 2 (not -2) spaces for indentation. |
||
puts "\n" + key.capitalize + " is " + recentPriceData["price"].to_s + " BYN in Minsk these days." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
minPrice = getMinPrice(products[key]) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Naming/VariableName: Use snake_case for variable names. |
||
puts "Lowest was on " + minPrice["year"] + "/" + monthMap[minPrice["month"]].to_s + " at price " + minPrice["price"].to_s + " BYN" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
maxPrice = getMaxPrice(products[key]) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Naming/VariableName: Use snake_case for variable names. |
||
puts "Maximum was on " + maxPrice["year"] + "/" + monthMap[maxPrice["month"]].to_s + " at price " + maxPrice["price"].to_s + " BYN" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
similarProducts = getSimilarPriceProducts(recentPriceData, products) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Naming/VariableName: Use snake_case for variable names. |
||
if similarProducts.empty? == 0 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/NumericPredicate: Use similarProducts.empty?.zero? instead of similarProducts.empty? == 0. |
||
puts "No products for similar price" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
else | ||
puts "For similar price you also can afford" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. |
||
puts similarProducts | ||
end | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Layout/BlockAlignment: } at 236, 8 is not aligned with keys.each { |key| at 222, 6. |
||
end | ||
end | ||
end | ||
|
||
main |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style/HashSyntax: Use the new Ruby 1.9 hash syntax.