Like all good systems we have to have a design. Because FOST.3™ features a UML compiler this becomes the natural medium to use to draw up the design and for me to use to think it through to start with.
Experience has taught me that single large databases aren't good for large data volumes and also that although in general keeping partial results and end results that can be calculated from the data is a bad idea, if the calculations are expensive then it is often actually a good idea.
Looking at it I decided to use three different database schemas that would all co-operate in order to handle not only the data analysis but also some of the project/operations management tasks¹ [1This project is not quite like the buisness systems that FOST.3™ normally targets in that the software development and execution is the project, rather than an enabler to the business operation that supports the business. Normally the software supports operations management, and the project management is there to support software development, but here the two terms amount to the same thing.].
The final two databases together will allow me to go back and double check results and look for mistakes, of which there are likely to be many.