mardi 18 août 2015

How to go about this web application project?

Before I start this project, I just want to make sure that everything I do is reasonable and efficient, so I thought getting criticism from fellow SO'ers would be a great idea :) I know that there are no "errors" yet, but I wouldn't want to rewrite the entire program again in case an error arises.

---PREFACE---(not necessary) I'm using league of legends game data received by their api to do some data analysis of matches. I want to know some big statistics(what's the average kill/death/assist ratio in north america for this character? When do people usually buy item x in a game? etc).

---GETTING JSON--- So I have a production API Key to call the league of legends API to get JSON data. Each time I get one package of json(very long, details of 1 single match) using java backend(using jersey for api cals) to call to limit my api call requests. Since Jersey's client is multithreaded, I will pass all the calls through a rate limiting synchronized block to make sure the amount of calls will not reach the limit.

---PROCESSING JSON---Idea 1 After I get the huge JSON object(I'm using org.json), I'm going to skip the majority of the data processing, except for gathering a few fields(let's say the kills/deaths/assists of all players and their item id's, champion id's, etc). Then, I'm going to create my own java beans to hold those data, and increment an already-existing bean inside the database(say I have a bean that has qty:5, kills:25, deaths:5, assists:10, then I can easily find that the avg kda is 5:1:2). Then I will throw the entire json object(which is really large) into the database.

PRO: very fast insertion process(no need to parse all the json and instantiate new things) CON: when I want to access specific things inside of the huge json, I would have to query for the entire match json in the database -> reading cost == high

---PROCESSING JSON---Idea 2 After I get the huge JSON object, I'm going to create three beans(1 for the match,1 for each player, and 1 for each team) and conquer and divide the json into 13 beans(10 player beans, 2 team beans, 1 match bean), and store them into the database. As each player bean is inserted, I will increment the already-existing bean with the k/d/a. The 13 beans will be linked by foreign keys(match id, player id, team id)

PRO: very organized and the json will be put into specific mongodb collections(1 for players, 1 for team, 1 for match) CON: very slow insertion since I would need to read a ton of json to create the objects. Might also have a ton of 0's since Riot API's json sometimes omits fields, and my beans will have those fields that it omits, leading to a ton of 0's in my database(or I could just make each bean a map with no generic? Is that good practice?)

---STORING JSON---Idea 1 continued Store everything in 1 collection(not really 1, but 1 for each region like North America, Western Europe, Eastern Europe, Asia, etc)

---STORING JSON---Idea 2 continued Store everything in its own collection.

The rest is just data analysis and sending the info to the frontend which I think there wouldn't be too much trouble. Is there any way of making my application faster?(an example would be putting indexes inside the mongodb collection)

For this kind of statistics displaying website, there will be a lot of people adding their own data into the database, but even more people are going to the website simply to see the statistics.

Thank you!

Aucun commentaire:

Enregistrer un commentaire