Graphing Prime Numbers

I enjoy toying with prime numbers and wanted to see how the Java GraphStream library worked.  Here is a short video of the result. 

 

 

Interestingly, it is little fun projects like this that often turn into moments of learning.  While working on some prime number related programs I learned of some tool limits I wasn't aware of before this.

•  The max radix value for converting numbers to strings is 36.  From some searching, it sounds like that was chosen by being the number of values 0-9 + the 26 case-insensitive values a-z.  I found it odd when numbers I converted starting at radix 37 were identical.
• Integer.toString(99999, 37)
  
•  For a call to: com.google.common.collect.Sets.powerSet( aSet) , the "aSet" set parameter is limited to a max of 30 items.

 

 

I decided to try a variety of layout parameters to look for interesting results.  Here are a few of those results using GraphStream.

 

 Those were interesting but I decided to see if I could integrate different graphing libraries to look for other interesting features, etc.

My first attempt was to try integrating gephi but that didn't go well.   As I looked into other graphing libraries, I noted that many are not maintained.  This took way too much effort due to the version of netbeans libraries that had been used.  I ended up forcing newer versions which are hosted by Apache.  Eventually, I had code that built but no visual graph was produced - I noted some flakiness with one of my displays when trying to run it so I suspect an issue between older code and my display/drivers.  

Next, I decided to try Jgrapht.  This had an initial snag which was the fact that the original package was no longer maintained and wasn't available via maven.  I ended up finding a version that is either a fork or repackaging of it and is available via maven.  Now, things were much less difficult.  It also resulted in very few total dependencies - even after I reworked the code significantly and added in the picocli lib for command line arg handling. I also ended up needing to add in the jgrapht-ext and jgraphx libs to display resulting graphs.

Here are a couple examples from that. The first uses the pre-canned compact tree layout. It took some playing to get something useful to display though. The second graph uses the pre-canned circle layout and that was easy to setup but still not as visually appealing as I desired.

I'll keep looking and trying new settings to see if I can come up with something better but this does work at least.  From a visual graph standpoint, it doesn't scale very well it seems but maybe some tuning is possible.  I got the generation of the primes tuned much better now and could generate 1M primes quickly but trying to graph even 10k primes seems unrealistic / painful.  I'll probably end up terminating the running process before it ever gets to the point of displaying the graph.

[update] I added the ability to export my data into the Graph Modelling Language (GML) format and that is compatible with the yed tool. Yed has much richer layout support that what I have been working with from a Java library integration standpoint.  The downside is that really large datasets are time consuming to visualize.  I'm not sure how long it took for yed to complete this since I took a several hour break.  This was for 10k primes.

From Microk8s to Usernetes - saga began

 I had a desire to work with more of the "rootless" technologies and found Microk8s a bit limiting.  So I started to look around at what else existed and what state of maturity everything was in.  From a maturity level, much of the work is still pretty new but after looking over information on a few projects, I decided to switch over to Usernetes.

If you review the page you will find a few alternatives and some of the various trade-offs.

The main options listed at the time were Kind, Minikube, Usernetes and K3s with experimental support.

My main initial draw to Usernetes is that it is the reference Kubernetes distribution for Rootless mode.

By this point, I had made some attempts to converge some of the various rootless related setup on my system and had broken some things pretty badly at times while trying to workout changes.  I finally just uninstalled Microk8s and started from that point.  

I was hoping I could still get my deployments from Eclipse working (including running my own private registry) and also keep the SpringBoot / ConfigMap integrations working.

Some of the initial challenges to plan for:

• Microk8s had some features/add-on options you can install which I needed to account for - the major ones being.
• DNS
• dashboard
• Ingress (Nginx)
• MetalLB
• Registry 
• Local Storage

The good news is;

•  DNS wasn't much of a problem.  
• I've not gotten to the dashboard yet.  
• Since I made a conscious choice to avoid Nginx as Ingress in Microk8s - it was less of an issue.  Really, the initial issue was how to avoid the default install of Traefik.  Once I figured that out, things improved a bit.
• I installed HAProxy/kubernetes-ingress just like I had in Microk8s. I still may need to work out some changes to settings yet - not quite clear yet.
  
• MetalLB installed ok - I've still got some items to sort out regarding accessing services from outside the cluster though.
• Helm3 installed without many issues.
• I was able to install cert-manager OK.
• I was able to get the registry installed but  am working through access issues from outside the cluster still.
• I was able to get my image for ldap up and running with some effort.
• I was able to get my Postgresql DB up and  running with some effort

Getting my LDAP and Postgresql images running took unexpected time. I made the conscious decision to work with cgroupv2 when I still Microk8s up but once I got the base Usernetes up, I found issues with accessing existing persistent volumes. The workaround for now was to use setfacl to set permissions in a few cases along with some chown / chmod work and some cluster restarts and other random bits which finally worked out.  I've got some more reading to do on cleaner ways to handle this but I got the images running at least.

I am still working out cluster external access to services - before I didn't have much trouble accessing the LDAP, Postgresql, and registry running in the cluster. Now I've got some extra hoops to sort out for external cluster access. It may come down to just needing calls to:

~/sw/usernetes/rootlessctl.sh add-ports

but I'm not 100% sure yet.  I'm still a bit early in the process.  

Also note that, just to different, I did the Usernetes install for CRIO instead of containerd.

    $ ./install.sh --cri=crio

Which may result in some differences too.

As it stands right now though; I seem to be on the right track:

scott@scott-z1:~$  systemctl --user status -T u7s.target
● u7s.target - Usernetes target (all components in the single node)
     Loaded: loaded (/home/scott/.config/systemd/user/u7s.target; enabled; vendor preset: enabled)
     Active: active since Fri 2021-09-24 08:57:08 EDT; 3 days ago

Sep 24 08:57:08 scott-z1 systemd[2398]: Reached target Usernetes target (all components in the single node).
 

My Podman and Docker are both running rootless at the moment as well but I've got various things tied to containerd. I'd like to simply that a bit more.

But as is, the cluster isn't taking up unrealistic resources.  Working to get logging reasonable with Microk8s and trying to simplify down to an IP4 versus dual stack has helped to a large degree.  A number of services generate quite a bit of logging related to the firewall. I've reduced that a little bit but still some work to go.

I'll have to write up some updates when I get the remaining issues smoothed out - hard to make the time with other commitments though.

Wishing you a blessed day!

Scott

[Update 2022/02/02] I've given up on Usernetes / k3s rootless for now.  The reason is only really related to the usermode networking - slirp4netns.  Maybe this will become easier to use over time in larger deployments but for now it seems a bit of hassle to integrate with existing technology, tools and designs.

Microk8s - minor anoyance

I've been working with Microk8s for a little while now and it generally works well. There is one area where it seems to fall short though - and that is in the area of configuration flexibility. 

I've been starting to work with other tools such as Podman, Buildah, Docker and rootless mode.  I've also been investigating various OCI & CRI runtimes - crun, runc, containerd, cri-o.  As part of that, I want to try the various configurations to see the differences myself.  

Kubernetes deprecated Docker as the runtime for versions 1.21+ and will use CRI in the form of containerd.  This is fine but I would also like to work with crun to determine if there are benefits for some use cases.  Most information I've run across indicates that crun requires less memory and tends to support new features first.  In the use case where I would like to run a good number of services on either a single node cluster or very small cluster as part of some research - crun might provide some benefits.  At the moment, It doesn't seem like configuring Microk8s with crun is an option.  Since OCI and CRI tend to promote the fact that alternative runtimes are supported - I'm somewhat surprised that configuring alternatives in Microk8s is either not supported or possibly just difficult.

I've not found a lot of good/detailed "how-to" articles regarding alternative configuration of many of the container/runtime related technologies. Regardless, I found this post indicating Microk8s lacks the ability to handle alternative setups but the post is old enough that something could have changed or at least the underlying reason may have.

This is a good reason to review other options at least.  I'd rather be able to decide certain tradeoffs based on my needs rather than not have the option. With all the flexibility that K8S provides in general, it seems unfortunate that individual "flavors" may limit configurations. I can see defaulting to some specific configurations - that provides a way to reduce initial complexity but removing/invalidating some configurations seems a bit too "embrace and extend" and/or "Either accept the propriety platform or don't use it" attitudish. I'm hoping I am wrong and can eventually find a way to test out some other configurations of interest.

John 17:3

World English Bible

³ This is eternal life, that they should know you, the only true God, and him whom you sent, Jesus Christ.