ruby on rails - Rake Task to parse JSON, some objects might be missing -
i made simple rake task parse json , i'm using apply attributes game collection.
to avoid massive list of json in block json can previewed here , here - these steam store's api returning game details. rake worked when pulling 2 pieces of information present on every single record (assuming success)
desc 'update steam game info' namespace :update_steam_games task scrape_info: :environment require 'open-uri' #require 'nokogiri' require 'json' games = steamgame.first(10) games.each |game| base_url = "http://store.steampowered.com/api/appdetails?appids=#{game['appid']}" puts "base_url: #{base_url}" results = open(base_url) { |f| result = json.parse(f.read) puts result.inspect next if result[game['appid'].to_s]['success'] == false type = result[game['appid'].to_s]['data']['type'] detailed_description = result[game['appid'].to_s]['data']['detailed_description'] about_the_game = result[game['appid'].to_s]['data']['about_the_game'] short_description = result[game['appid'].to_s]['data']['short_description'] supported_languages = result[game['appid'].to_s]['data']['supported_languages'] if !result[game['appid'].to_s]['data']['dlc'].nil? dlc = result[game['appid'].to_s]['data']['dlc'].flatten end is_free = result[game['appid'].to_s]['data']['is_free'] required_age = result[game['appid'].to_s]['data']['required_age'] genres = result[game['appid'].to_s]['data']['genres'].map { |g| g['description'] } developers = result[game['appid'].to_s]['data']['developers'] publishers = result[game['appid'].to_s]['data']['publishers'] windows = result[game['appid'].to_s]['data']['platforms']['windows'] mac = result[game['appid'].to_s]['data']['platforms']['mac'] metacritic_score = result[game['appid'].to_s]['data']['metacritic']['score'] metacritic_link = result[game['appid'].to_s]['data']['metacritic']['url'] #game.update_attributes(genres: genres, detailed_description: detailed_description) puts game.inspect } end end end
the current error i'm getting nomethoderror: undefined method
[]' nil:nilclass` on line 34 - end of attribute assignment, not sure it's directly related line.
i did realize trying store boolean integer , have since fixed it, getting error.
i've tried wrapping if xxx .present?
wrapper, below
desc 'update steam game info' namespace :update_steam_games task scrape_info: :environment require 'open-uri' #require 'nokogiri' require 'json' games = steamgame.first(10) games.each |game| base_url = "http://store.steampowered.com/api/appdetails?appids=#{game['appid']}" puts "base_url: #{base_url}" results = open(base_url) { |f| result = json.parse(f.read) puts result.inspect next if result[game['appid'].to_s]['success'] == false if type = result[game['appid'].to_s]['data']['type'].present? type = result[game['appid'].to_s]['data']['type'] end if detailed_description = result[game['appid'].to_s]['data']['detailed_description'].present? detailed_description = result[game['appid'].to_s]['data']['detailed_description'] end if about_the_game = result[game['appid'].to_s]['data']['about_the_game'].present? about_the_game = result[game['appid'].to_s]['data']['about_the_game'] end if short_description = result[game['appid'].to_s]['data']['short_description'].present? short_description = result[game['appid'].to_s]['data']['short_description'] end if supported_languages = result[game['appid'].to_s]['data']['supported_languages'].present? supported_languages = result[game['appid'].to_s]['data']['supported_languages'] end if result[game['appid'].to_s]['data']['dlc'].present?.present? dlc = result[game['appid'].to_s]['data']['dlc'].flatten end if is_free = result[game['appid'].to_s]['data']['is_free'].present? is_free = result[game['appid'].to_s]['data']['is_free'] end if required_age = result[game['appid'].to_s]['data']['required_age'].present? required_age = result[game['appid'].to_s]['data']['required_age'] end if genres = result[game['appid'].to_s]['data']['genres'].map { |g| g['description'] }.present? genres = result[game['appid'].to_s]['data']['genres'].map { |g| g['description'] } end if developers = result[game['appid'].to_s]['data']['developers'].present? developers = result[game['appid'].to_s]['data']['developers'] end if publishers = result[game['appid'].to_s]['data']['publishers'].present? publishers = result[game['appid'].to_s]['data']['publishers'] end if windows = result[game['appid'].to_s]['data']['platforms']['windows'].present? windows = result[game['appid'].to_s]['data']['platforms']['windows'] end if mac = result[game['appid'].to_s]['data']['platforms']['mac'].present? mac = result[game['appid'].to_s]['data']['platforms']['mac'] end if metacritic_score = result[game['appid'].to_s]['data']['metacritic']['score'].present? metacritic_score = result[game['appid'].to_s]['data']['metacritic']['score'] end if metacritic_link = result[game['appid'].to_s]['data']['metacritic']['url'].present? metacritic_link = result[game['appid'].to_s]['data']['metacritic']['url'] end #game.update_attributes(genres: genres, detailed_description: detailed_description) puts game.inspect } end end end
but still end same issue, in addition fact i'm unsure how i'm going pass game.update_attributes
without knowing if exists.
how inspect problem, , make less redundant?
the json
in case don't want click link have provided sample of json here. have removed of bloat. 10
starts dynamic id.
"10": { "success": true, "data": { "type": "game", "name": "counter-strike", "steam_appid": 10, "required_age": 0, "is_free": false, "detailed_description": "play world's number 1 online action game. engage in incredibly realistic brand of terrorist warfare in wildly popular team-based game. ally teammates complete strategic missions. take out enemy sites. rescue hostages. role affects team's success. team's success affects role.", "about_the_game": "play world's number 1 online action game. engage in incredibly realistic brand of terrorist warfare in wildly popular team-based game. ally teammates complete strategic missions. take out enemy sites. rescue hostages. role affects team's success. team's success affects role.", "short_description": "", "supported_languages": "english, french, german, italian, spanish, simplified chinese, traditional chinese, korean", "header_image": "http://cdn.akamai.steamstatic.com/steam/apps/10/header.jpg?t=1447887426", "website": null, "pc_requirements": { "minimum": "\n\t\t\t<p><strong>minimum:</strong> 500 mhz processor, 96mb ram, 16mb video card, windows xp, mouse, keyboard, internet connection<br /></p>\n\t\t\t<p><strong>recommended:</strong> 800 mhz processor, 128mb ram, 32mb+ video card, windows xp, mouse, keyboard, internet connection<br /></p>\n\t\t\t" }, "mac_requirements": { "minimum": "minimum: os x snow leopard 10.6.3, 1gb ram, 4gb hard drive space,nvidia geforce 8 or higher, ati x1600 or higher, or intel hd 3000 or higher mouse, keyboard, internet connection" }, "linux_requirements": { "minimum": "minimum: linux ubuntu 12.04, dual-core intel or amd @ 2.8 ghz, 1gb memory, nvidia geforce 8600/9600gt, ati/amd radeaon hd2600/3600 (graphic drivers: nvidia 310, amd 12.11), opengl 2.1, 4gb hard drive space, openal compatible sound card" }, "developers": [ "valve" ], "publishers": [ "valve" ], "price_overview": { "currency": "usd", "initial": 999, "final": 999, "discount_percent": 0 }, "packages": [ 7 ], "package_groups": [ { "name": "default", "title": "buy counter-strike", "description": "", "selection_text": "select purchase option", "save_text": "", "display_type": 0, "is_recurring_subscription": "false", "subs": [ { "packageid": 7, "percent_savings_text": "", "percent_savings": 0, "option_text": "counter-strike: condition 0 - $9.99", "option_description": "", "can_get_free_license": "0", "is_free_license": false, "price_in_cents_with_discount": 999 } ] } ], "platforms": { "windows": true, "mac": true, "linux": true }, "metacritic": { "score": 88, "url": "http://www.metacritic.com/game/pc/counter-strike?ftag=mcd-06-10aaa1f" } } }
update
i realized can't check on existance of platform - windows
if platform
doesn't exist. code executes, however, having issue writing since lot of parameters may or may not missing on every pass.
what proper method instead of if
blocks everywhere?
update
my code works after realize stupidly put if
block around variable definement portion.
however, still redundant. there proper way this, or fine?
games = steamgame.first(10) games.each |game| base_url = "http://store.steampowered.com/api/appdetails?appids=#{game['appid']}" puts "base_url: #{base_url}" results = open(base_url) { |f| result = json.parse(f.read) game_appid = game[:appid].to_s result_for_appid = result[game_appid] result_data = result_for_appid[:data] puts result.inspect next if result_for_appid[:success] == false type = result_data[:type] detailed_description = result_data[:detailed_description] about_the_game = result_data[:about_the_game] short_description = result_data[:short_description] supported_languages = result_data[:supported_languages] if result_data[:dlc].present? dlc = result_data[:dlc].flatten end is_free = result_data[:is_free] required_age = result_data[:required_age] # genres = result_data[:genres].map { |g| g[:description] } # provided json didn't contain 'genres' inside 'data' developers = result_data[:developers] publishers = result_data[:publishers] windows = result_data[:platforms][:windows] mac = result_data[:platforms][:mac] metacritic_score = result_data[:metacritic][:score] metacritic_link = result_data[:metacritic][:url] # game.update_attributes(genres: genres, detailed_description: detailed_description) # provided json didn't contain 'genres' inside 'data' no update game.update_attributes(detailed_description: detailed_description) puts game.inspect } end end
few tips this: dry - don't repeat can access json keys symbols didn't have "genres' in provided json don't have use if's - when there's no data get: variable = nil. can later update attributes regarding variables values.
Comments
Post a Comment