Application Design

[RobMunfy]

After reading a post on here, I've been inspired to build an application myself. However, I'm struggling to think of the ideal design for it. The application will be written purely in Java.

The application consists of three parts:
1) A Spring MVC web based front-end
2) A selenium based web scraper that will be run every 5 hours or so
3) A database to store the data the web scraper retrieves

So I figured that these 3 different parts should be hosted on 3 different servers ultimately. However, I then realised that the web front-end will need to connect to the database and so will the web scraper. In that case I will want the model part of MVC and the objects created in the web scraper to be the same. I guess in that case what I am after is some sort of shared model between the web scraper and the web front-end.

Any help with the best possible design for this?

 

[GravyMonster]

There's no reason for 3 servers, unless you want to ruin your bank account

I've not done anything web-based with Java, but it sounds like your web scraper can be run from a CRON job/scheduled task from the server. The database can also be stored on the server itself. Any half-decent hosting package should be capable of these functions.

 

[touch]

There's no reason for 3 servers, unless you want to ruin your bank account

This. It doesnt really make a difference if everything is run from the same server or not. The code will be the same either way, the only difference will be the connection string to the database will be localhost rather than an external IP address.

Design the database first and the classes and attributes for each part of the application will be based on those and will naturally be the same. You dont need to worry about having the model for each part being the same. They wont communicate directly with each other, just with the database.