Database Pool

From KallestadWiki

The thought has occurred to me that connection pooling and threading would be a very efficient way of handling database interactions with KDF. It would also keep the code base a little bit cleaner as the database specific code would reside outside the main KDF code set.

There are several thoughts that immediately come to mind. First off, I don't know that DBD is thread safe, and if it is, I'm not sure all the DBI::xxxx drivers would be thread safe. Spawning an untold number of processes isn't a good idea either - especially when the primary connection initiation mechanism would be a web server. I'll look into it a bit more, but right now, I'm leaning towards a POE based connection pooling mechanism.

Other thoughts that strike me at this time are as follows:

• keep KDF Abstraction out of the equation so that the pooling mechanism can be used for code that is not based on KDF
• Design should be such that the pooling mechanism is not database dependent, nor is it dependent on SQL support. I don't know that this can be done without having a lot of potentially unnecessary code, but I do think support for Lucene interaction is important, and I would like to be able to easily integrate KDF with a variety of storage mechanisms and I want as few restrictions as possible for those storage mechanisms.
• Cooperative IPC for building result sets off of multiple data sources. Again, I'm not sure about implementation, but Lucene comes to mind in that you may want to query lucene for it's full text capabilities and thin the result set further based on SQL data.
• Query cacheing via libmemcached or other shared memory cacheing mechanisms. Databases do their own cacheing, but application developers know better about which queries really should be hitting the system the hardest, and about the volatility of those result sets. The decision still needs to be made as to whether the cache mechanism should reside within the application framework or the data interaction mechanism.

A connection pool is really just that. You would use a connection pool in order to cache data source connections so as not to require the application to waste resources opening and connecting to data sources that it uses with a great deal of regularity. The CPU cycles required for connectivity are thin, but when you connect frequently the CPU cycles can add up dramatically.

Database Connection Pooling Topics

• Database Pool
• Development Concerns
  • Thread Safety
  • POE
  • POE DB Modules
• Design Overview

Kallestad Development Framework Topics

• KDF Home
• Project Overview
  • Peripheral Objectives
• Technical Information
  • Back End
    • Complex Objects
    • Abstract Elements
    • SQL API
  • Front End
  • Database Pool
• Licensing