ruby on rails - Rake Task to parse JSON, some objects might be missing -


i made simple rake task parse json , i'm using apply attributes game collection.

to avoid massive list of json in block json can previewed here , here - these steam store's api returning game details. rake worked when pulling 2 pieces of information present on every single record (assuming success)

desc 'update steam game info'  namespace :update_steam_games      task scrape_info: :environment      require 'open-uri'     #require 'nokogiri'     require 'json'      games = steamgame.first(10)     games.each |game|         base_url = "http://store.steampowered.com/api/appdetails?appids=#{game['appid']}"         puts "base_url: #{base_url}"         results = open(base_url) { |f|             result = json.parse(f.read)             puts result.inspect             next if result[game['appid'].to_s]['success'] == false             type = result[game['appid'].to_s]['data']['type']             detailed_description = result[game['appid'].to_s]['data']['detailed_description']             about_the_game = result[game['appid'].to_s]['data']['about_the_game']             short_description = result[game['appid'].to_s]['data']['short_description']             supported_languages = result[game['appid'].to_s]['data']['supported_languages']             if !result[game['appid'].to_s]['data']['dlc'].nil?                 dlc = result[game['appid'].to_s]['data']['dlc'].flatten             end             is_free = result[game['appid'].to_s]['data']['is_free']             required_age = result[game['appid'].to_s]['data']['required_age']             genres = result[game['appid'].to_s]['data']['genres'].map { |g| g['description'] }             developers = result[game['appid'].to_s]['data']['developers']             publishers = result[game['appid'].to_s]['data']['publishers']             windows = result[game['appid'].to_s]['data']['platforms']['windows']             mac = result[game['appid'].to_s]['data']['platforms']['mac']             metacritic_score = result[game['appid'].to_s]['data']['metacritic']['score']             metacritic_link = result[game['appid'].to_s]['data']['metacritic']['url']             #game.update_attributes(genres: genres, detailed_description: detailed_description)              puts game.inspect         }         end     end end 

the current error i'm getting nomethoderror: undefined method[]' nil:nilclass` on line 34 - end of attribute assignment, not sure it's directly related line.

i did realize trying store boolean integer , have since fixed it, getting error.

i've tried wrapping if xxx .present? wrapper, below

desc 'update steam game info'  namespace :update_steam_games      task scrape_info: :environment      require 'open-uri'     #require 'nokogiri'     require 'json'      games = steamgame.first(10)     games.each |game|         base_url = "http://store.steampowered.com/api/appdetails?appids=#{game['appid']}"         puts "base_url: #{base_url}"         results = open(base_url) { |f|             result = json.parse(f.read)             puts result.inspect             next if result[game['appid'].to_s]['success'] == false             if type = result[game['appid'].to_s]['data']['type'].present?                 type = result[game['appid'].to_s]['data']['type']             end             if detailed_description = result[game['appid'].to_s]['data']['detailed_description'].present?                 detailed_description = result[game['appid'].to_s]['data']['detailed_description']             end             if about_the_game = result[game['appid'].to_s]['data']['about_the_game'].present?                 about_the_game = result[game['appid'].to_s]['data']['about_the_game']             end             if short_description = result[game['appid'].to_s]['data']['short_description'].present?                 short_description = result[game['appid'].to_s]['data']['short_description']             end             if supported_languages = result[game['appid'].to_s]['data']['supported_languages'].present?                 supported_languages = result[game['appid'].to_s]['data']['supported_languages']             end             if result[game['appid'].to_s]['data']['dlc'].present?.present?                     dlc = result[game['appid'].to_s]['data']['dlc'].flatten             end             if is_free = result[game['appid'].to_s]['data']['is_free'].present?                 is_free = result[game['appid'].to_s]['data']['is_free']             end             if required_age = result[game['appid'].to_s]['data']['required_age'].present?                 required_age = result[game['appid'].to_s]['data']['required_age']             end             if genres = result[game['appid'].to_s]['data']['genres'].map { |g| g['description'] }.present?                 genres = result[game['appid'].to_s]['data']['genres'].map { |g| g['description'] }             end             if developers = result[game['appid'].to_s]['data']['developers'].present?                 developers = result[game['appid'].to_s]['data']['developers']             end             if publishers = result[game['appid'].to_s]['data']['publishers'].present?                 publishers = result[game['appid'].to_s]['data']['publishers']             end             if windows = result[game['appid'].to_s]['data']['platforms']['windows'].present?                 windows = result[game['appid'].to_s]['data']['platforms']['windows']             end             if mac = result[game['appid'].to_s]['data']['platforms']['mac'].present?                 mac = result[game['appid'].to_s]['data']['platforms']['mac']             end             if metacritic_score = result[game['appid'].to_s]['data']['metacritic']['score'].present?                 metacritic_score = result[game['appid'].to_s]['data']['metacritic']['score']             end             if metacritic_link = result[game['appid'].to_s]['data']['metacritic']['url'].present?                 metacritic_link = result[game['appid'].to_s]['data']['metacritic']['url']             end             #game.update_attributes(genres: genres, detailed_description: detailed_description)              puts game.inspect         }         end     end end 

but still end same issue, in addition fact i'm unsure how i'm going pass game.update_attributes without knowing if exists.

how inspect problem, , make less redundant?

the json

in case don't want click link have provided sample of json here. have removed of bloat. 10 starts dynamic id.

  "10": {     "success": true,     "data": {       "type": "game",       "name": "counter-strike",       "steam_appid": 10,       "required_age": 0,       "is_free": false,       "detailed_description": "play world's number 1 online action game. engage in incredibly realistic brand of terrorist warfare in wildly popular team-based game. ally teammates complete strategic missions. take out enemy sites. rescue hostages. role affects team's success. team's success affects role.",       "about_the_game": "play world's number 1 online action game. engage in incredibly realistic brand of terrorist warfare in wildly popular team-based game. ally teammates complete strategic missions. take out enemy sites. rescue hostages. role affects team's success. team's success affects role.",       "short_description": "",       "supported_languages": "english, french, german, italian, spanish, simplified chinese, traditional chinese, korean",       "header_image": "http://cdn.akamai.steamstatic.com/steam/apps/10/header.jpg?t=1447887426",       "website": null,       "pc_requirements": {         "minimum": "\n\t\t\t<p><strong>minimum:</strong> 500 mhz processor, 96mb ram, 16mb video card, windows xp, mouse, keyboard, internet connection<br /></p>\n\t\t\t<p><strong>recommended:</strong> 800 mhz processor, 128mb ram, 32mb+ video card, windows xp, mouse, keyboard, internet connection<br /></p>\n\t\t\t"       },       "mac_requirements": {         "minimum": "minimum: os x  snow leopard 10.6.3, 1gb ram, 4gb hard drive space,nvidia geforce 8 or higher, ati x1600 or higher, or intel hd 3000 or higher mouse, keyboard, internet connection"       },       "linux_requirements": {         "minimum": "minimum: linux ubuntu 12.04, dual-core intel or amd @ 2.8 ghz, 1gb memory, nvidia geforce 8600/9600gt, ati/amd radeaon hd2600/3600 (graphic drivers: nvidia 310, amd 12.11), opengl 2.1, 4gb hard drive space, openal compatible sound card"       },       "developers": [         "valve"       ],       "publishers": [         "valve"       ],       "price_overview": {         "currency": "usd",         "initial": 999,         "final": 999,         "discount_percent": 0       },       "packages": [         7       ],       "package_groups": [         {           "name": "default",           "title": "buy counter-strike",           "description": "",           "selection_text": "select purchase option",           "save_text": "",           "display_type": 0,           "is_recurring_subscription": "false",           "subs": [             {               "packageid": 7,               "percent_savings_text": "",               "percent_savings": 0,               "option_text": "counter-strike: condition 0 - $9.99",               "option_description": "",               "can_get_free_license": "0",               "is_free_license": false,               "price_in_cents_with_discount": 999             }           ]         }       ],       "platforms": {         "windows": true,         "mac": true,         "linux": true       },       "metacritic": {         "score": 88,         "url": "http://www.metacritic.com/game/pc/counter-strike?ftag=mcd-06-10aaa1f"       }     } } 

update

i realized can't check on existance of platform - windows if platform doesn't exist. code executes, however, having issue writing since lot of parameters may or may not missing on every pass.

what proper method instead of if blocks everywhere?

update

my code works after realize stupidly put if block around variable definement portion.

however, still redundant. there proper way this, or fine?

games = steamgame.first(10) games.each |game|     base_url = "http://store.steampowered.com/api/appdetails?appids=#{game['appid']}"     puts "base_url: #{base_url}"     results = open(base_url) { |f|         result = json.parse(f.read)         game_appid = game[:appid].to_s         result_for_appid = result[game_appid]         result_data = result_for_appid[:data]         puts result.inspect         next if result_for_appid[:success] == false         type = result_data[:type]         detailed_description = result_data[:detailed_description]         about_the_game = result_data[:about_the_game]         short_description = result_data[:short_description]         supported_languages = result_data[:supported_languages]         if result_data[:dlc].present?             dlc = result_data[:dlc].flatten         end         is_free = result_data[:is_free]         required_age = result_data[:required_age]         # genres = result_data[:genres].map { |g| g[:description] }         # provided json didn't contain 'genres' inside 'data'         developers = result_data[:developers]         publishers = result_data[:publishers]         windows = result_data[:platforms][:windows]         mac = result_data[:platforms][:mac]         metacritic_score = result_data[:metacritic][:score]         metacritic_link = result_data[:metacritic][:url]         # game.update_attributes(genres: genres, detailed_description: detailed_description)         # provided json didn't contain 'genres' inside 'data' no update         game.update_attributes(detailed_description: detailed_description)          puts game.inspect     }     end end 

few tips this: dry - don't repeat can access json keys symbols didn't have "genres' in provided json don't have use if's - when there's no data get: variable = nil. can later update attributes regarding variables values.


Comments

Popular posts from this blog

node.js - Node js - Trying to send POST request, but it is not loading javascript content -

javascript - Replicate keyboard event with html button -

javascript - Web audio api 5.1 surround example not working in firefox -