This is a short demo of running a SpringBoot batch job in eclipse where the configuration is retrieved from a Microk8s configmap. The setup can be extended to use K8s secrets but wasn't done here yet.
I had worked with several other tools when initially trying to import this data into my database but I ran into a number of problems with the methods. The biggest issue is some "complex" data as part of a set of fields that most import tools/processes don't handle correctly but where handled here fine. It did take a few iterations to work out a few incorrect field data types but Spring Batch provides pretty clear information regarding the problematic fields and types.
Here you can see an example of what wasn't handled well by some other import methods. It usually included partially quoted data and also commas.
Here is a quick screen shot of the Microk8s info showing my Postgresql database. My batch job uses the service name as the host portion of the connection string. For now, I manually update my DNS resolutions to map the service name to the service IP from outside of my cluster. I'm working toward that being an integrated/automated item requiring no manual intervention.
Thanks for looking and have a blessed day!
Scott
No comments:
Post a Comment