Application Architecture and Design (and cloud aspects)

 Let's say you work for some organization and there is a need for a new application.  If you are responsible for the architecture / design then 

My initial post was going to end up being a "ancient" IT history book with some extra stuff. I decided that wasn't very helpful - not many people want to read a book in a "blog format".

I'm condensing the content down to the following. 

• Use standard practices for
• project management
• design techniques
• Understand the reasons/basis for soft/hard requirements - This covers why a requirement exist and why a specific requirement was chosen over other similar prospective requirements.
• Examples reasons are;
• development / operational / infrastructure costs
• time to market /  speed of handling requirement changes
• maintainability
• performance / scalability / availability
• Carefully evaluate technology choices.
• Consider the risk of using any technology that could become unsupported before the end of the useful life of your application. New "amazing" technologies may only be around for a short period of time before something new replaces it.
• Make plans to keep 3rd party frameworks, libraries and languages/run-times up-to-date with regard to the most current versions.
• Consider the risk, for example, of using early versions of SpringBoot such as 2.1 which are not maintained now - if a new security flaw comes up, it may be very hard to fix the issue.

With regard to standard practices, use whatever is appropriate for your organization and the project need. Many organizations are trying follow "Agile" practices but that doesn't mean that "Waterfall" is always bad to use. Use what is appropriate for the situation.

Understanding the reasons used for requirement selection helps to keep requirements aligned with each other and provides useful context for handling business changes over time.

Technology comes and goes.  Don't love or hate particular technology - it is a tool.  Use appropriate technology and if needs change then consider more appropriate technology.  Don't select technology just because it is considered "great" by any person or group - consider the positive and negative aspects in-depth for your use-case.  

Small example of considerations - given a service expecting high transaction rates; what technology stacks might be considered and what are the trade-offs?

1. Python
1. Positives
1. Increasingly used in enterprises
2. OpenSource
2. Negatives
1. Library ecosystem isn't as mature as for Java
1. You want a library to help with web requests?  Which option do you pick? What version of Python are you on?
1. https://pypi.org/search/?q=web+requests&page=1
3. Considerations
1. Security
1. https://www.theregister.com/2021/07/28/python_pypi_security/
2. https://medium.com/ochrona/arbitrary-code-execution-during-python-package-installation-3a60990350ef
3. https://snyk.io/blog/python-security-best-practices-cheat-sheet/
2. Some libraries need local compilation
3. Cross-version compatibility
4. Transitive dependency handling
5. Long term maintenance questions
1. https://medium.com/codex/python-4-0-will-never-arrive-3d994dce54f1
6. Runtime aspects
1. Python global lock
1. https://wiki.python.org/moin/GlobalInterpreterLock
2. Memory management - also consider context including containers
1. https://www.askpython.com/python/examples/memory-management-in-python
3. Threading
1. https://docs.python.org/3/library/threading.html
2. Java
1. Positives
1. Heavily used in enterprises
2. Generally has useful features and reasonably performance
3. Many enterprise level libraries and frameworks
2. Negatives
1. New version every 6 months by Oracle can cause some thrashing of work
2. OpenSource vs Oracle licensing
3. Considerations
1. Alternative JDK / JVM / run-times
1. Oracle JRE
2. OpenJDK
3. Azul
4. IBM OpenJ9
5. Eclipse Temurin
6. Amazon Corretto
2. SpringBoot vs JEE
3. Cross-version compatibility
4. Transitive dependency handling

I highly recommend reading about and understanding the runtime aspects of Python. Consider the effects of memory pressure within and across container instances (i.e. Docker, Kubernetes, etc) - especially in a cloud environment and / or where you are cost / latency sensitive. 

Regarding compatibility and transitive dependency handling - I've had people with architect and team lead titles make comments to me along the lines of:

• "Java versions are backward compatible so keeping up with current versions isn't important."
• "We are pinning the SpringBoot version at version 2.3 instead of upgrading to 2.7+ because it improves stability by preventing changes."

I believe these statements reflect some general misunderstandings or lack of knowledge. There are very instances of 100% cross-version compatibility for operating systems or programming languages over significant spans of time. You also have to clarify whether you are talking about things like source code / binary compatibility and forward / backward version compatibility. Try running a bunch of games written for Windows 95 on Windows 11 - how many will work?  In my opinion - likely few.  For operating systems, computer languages and frameworks - features are added and deprecated / removed across releases.  API's change due to needs such as performance, security, new features, etc. If OpenSource is brought up as a solution to these type of issues - I'll claim it isn't reasonable.  There are support matrices for most major OSs, computer languages and library / frameworks - they don't maintain old versions forever. If you are outside a support matrix - you have no guarantee that anything will work and getting help will likely be somewhere between hard and impossible. Needing help for a major security issue or something that breaks a critical system is not the time you want to find out that no one can help you or can't help you quickly. Some common support / compatibility matrices below..

• OpenJDK support matrix from RedHat
• https://access.redhat.com/articles/1299013
• SpringBoot support matrix
• https://spring.io/projects/spring-boot#support
• SpringBoot / SpringCloud compatibility matrix
• https://spring.io/projects/spring-cloud#overview
• Django support matrix / roadmap
• https://www.djangoproject.com/download/
• Nx / NodeJS / TypeScript compatibility matrix
• https://nx.dev/packages/workspace/documents/nx-nodejs-typescript-version-matrix

Thanks,

Scott

Working and living around Tampa - the challenges

It has been over six years since we moved from up north down to the Tampa area.  We lived in an apartment for a while when we first moved down here - the biggest reason for that was the time it afforded us to research and find the best place to buy a home.  

I did a lot of research before picking the Tampa area as the place we should move to.  In many ways that research was correct.  There are many nice things to do around Tampa, there are many IT jobs, etc.  When we were choosing a house though, I wanted to keep options open a bit though.  I didn't know people who worked at any of the local IT employers so I wanted to be located somewhere central in case a future employment change was needed.  This was very sound reasoning and in the end was useful to a degree.

The first place I worked at down here was located in Temple Terrace.  From our house, it was typically a 35-50 minute drive which wasn't too bad.  There were occasional days where it was over 1.5 hours when accidents occurred at the wrong place/time.  Overall this was fine - until that employer announced pending layoffs and everyone had 3 basic choices - quit on your own terms, stay and see if you ended up laid off, select to take a termination package which required you to stay for some amount of time before you could get their "layoff benefits" package.  Since I had only been there for a couple years, the "layoff benefits" package wasn't very enticing. If you ended up having to stay until one of a set of predetermined dates - you end up looking for work with a flood of other people at the same time. In the end, it seemed like the best move was to quit on my own terms and find a new job before a large mass of other people were also looking for work.

I didn't have a lot of time to find something else but I did my best.  I found a place which was starting some similar cloud work and had some other interesting aspects.  When I interviewed there (about 10am), the drive was only about 1 hour which wasn't bad.  I was a bit nervous because the distance was quite a bit farther and required taking the interstate which had traffic issues on the news almost daily.  I decided to take this job and nearly immediately found that it was typically 75-90 minutes driving there and usually at least 90 minutes coming home.  My GPS would try to route me around traffic but that was usually pointless - I often ended up on streets with stop signs every few feet or the route was just erratic.   I continued with this for around 8 months and was very tired of the drive.  It was hard to be home at a reasonable time to have a family dinner together.  It would have been better if they allowed some remote work but that was never an option.

I was contacted by someone from another company at around the 7 month mark and we started talking.  They were looking for someone which had  had prior development lead experience and lots of general development experience, etc overall. I would have some architecture, dev lead, mentoring, etc work which was pretty much the description of a "dream job" for me.  The routes to this company followed the same initial path as my current company at that time but the new company was closer - not far from the airport. I figured this would reduce drive time by 15-30 minutes one way.  So given the "dream job" option and less driving - it seemed like a good choice to make.

So I made that change.  It was a little less driving but it was still daily stop and go traffic with motorcycles driving between lanes and other just bizarre behavior.  It was usually between 50 and 75 minutes to work and 60-75 minutes home. Nearing the 2 year mark here and then COVID hit.  We had just moved many people, including myself, into a new building next door - all setup with the open floor plan and everyone sits together at groups of long desks.  A bunch of different things just made it feel like it was time to leave there.  The original "dream job" description didn't include all the red tape that affected everyone living in my household due to the nature of the business.  At this point, I really am worn out from the daily drives and traffic, etc.  I realize at this point that it is hard to find something "close" to where I live which is under an hour drive. 

I reach out to agencies and look for remote work.  Jobs which require coming into the office on a limited basis would be great.  I do enjoy the direct contact with others.  I also decide to go with contract or contract to hire positions for the first time ever.  If something changes for the worst, it seemed like it would simplify changing jobs.  

I find a job with a large well known company and it is contract to hire.  It is fully remote at that point but it was indicated that it could change a little.  If I had to go into the office it would be 1.5 to 2.5 hours one way but if that was only once a month or so then that is doable.  There were no real decisions on the details or plans at that point.  I decided to give that a go.  It was going ok but I wasn't doing any real design/architecture work and was pretty bored.  Then one day, it was announced that there would be a 3 day in-office requirement starting in 2 weeks.  That was obviously not going to work well for me and the 2 week notice for it was beyond strange.  Fortunately, my local manager had already requested an exception for a month for that in-office requirement for me.  That ended up extended for another month since they really liked what I was doing.  I knew that at some point that would have to end - if for nothing else, it isn't fair to other employees who have to go in.  At some point, it would cause some friction with other employees if I was left exempted from that requirement.  

Anyways, I finally reached an end-date there and now it is time to find something again which I would love to be a long term employer which doesn't require large numbers of hours on the road each month.

So my recommendation to anyone thinking about moving to a new area like Tampa - think carefully about where the jobs are versus where you want to live.  Don't underestimate the effects of drive time and traffic.  Also remember to account for other things though like floods, hurricanes, sink holes , etc too though..  In the end, it may cause you to rethink how you define "best location" to live.

Wishing everyone in similar circumstances the best!

Scott

Hurricane Ian and my parents home

 My parents have been in a nice retirement community for a few years now and this isn't the first hurricane but it is the only one that did significant damage to their home.  Total loss..  

Both my parents and my family are blessed though - no one was hurt and God takes care of the needs. 

I'm very happy we have enough space for them to move in and store many of their belongings while they work through the insurance and wait for a new home.  It is estimated as a good number of months before it will be ready.  It is hard to say if there will be other delays - there are so many people needing replacements now.

It was much different seeing their pictures compared to us being their when I took these. These are after a couple weeks so the roads were clear of debris and the water had receded from the roads we use to get there.

These are from a golf cart on our way to their house in the community.

The devastation was hard to comprehend at this scale.  Tornadoes tear up homes but that is usually in a path you can follow and it may skip areas a bit.  This was just destruction everywhere.  Not every home was a total loss but many were and most others had damage to some extent. 

This is what we found when we got to my parents home.

These next 2 pictures are of what used to be part of the lanai. Roof and most walls gone.  A refrigerator was tossed out in the yard. 

Something like 2/3 of the roof was gone - it was now "brighter inside" but mosquitoes were starting to multiply and lack of electric or any chance of usable air conditioning makes cleanup even harder. I can't say that the smell of wet insulation improves with time. There was some humor though as a few containers in awkward places were full of water - my wife can attest to that!  I was fortunate to not drop the container of fishing weights and water - but I did tell my parents that it made up for them helping me move many heavy books between homes over the years! I probably am off a bit on that - I'm sure they still moved more books than I moved wet fishing stuff.

It is in God's hands and nothing else matters.

Romans 15:13

13 May the God of hope fill you with all joy and peace as you trust in him, so that you may overflow with hope by the power of the Holy Spirit.

  Jeremiah 10:12-13

12 But God made the earth by his power; he founded the world by his wisdom and stretched out the heavens by his understanding.

13 When he thunders, the waters in the heavens roar; he makes clouds rise from the ends of the earth. He sends lightning with the rain and brings out the wind from his storehouses.

Psalms 71:18

18 Even when I am old and gray, do not forsake me, my God, till I declare your power to the next generation, your mighty acts to all who are to come.