Image from Pixabay
In this article, I will document and keep updating routine problems I face as a software engineer with proposed solutions I used to fix them.
This section lists troubleshooting I faced during dev projects.
Troubleshooting from my university enrollment sections app are listed next.
An Illegal Reflective Access Operation has Occurred in Apache POI
This was when Java 9 came out, and reflection operations were affected.
Apache POI 3.x is quite old, while 4.x came later.
The solution was to update Apache POI to a new version to get rid of the warning, as per my StackOverflow Question.
Troubleshooting from course projects are listed next.
Distributed Text File System
Documentation: Distributed Text File System | CP | DEV.
- Binary Incompatibility Issues: When refactoring app package name from
com.github.tobiasbriones, and loading older serialized objects from FS.
- Storage: When the linux container running a Java app runs out of space,
java.io.IOException: No space left on device.
- Fix IntelliJ Gradle JVM: I needed to change the Gradle JVM version used by IntellJ IDEA build tools to the correct version.
I list random troubleshooting experiences I face next.
This covers day-to-day annoying issues I have solved before in software development.
Kotlin toList Method not Found
The problem complains
Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
since the version Kotlin is running doesn’t have the method
Stream API that was introduced in Java 16.
The IDE was all right with the code, but it didn’t compile in the end.
A solution was to add
import kotlin.streams.toList to import the method
toList so it’s found. The problem is that is unnecessary, and the IDE
deleted it because it said it was an unused import.
There’s always a conflict in JVM setups of what you’re using: the dependency of the IDE for Java and JVM projects is evil.
I checked all possible configurations in the IDE, and everything was set up to JDK 19 (this code used to work before without setting up anything 😣 but it stopped compiling some day).
In the end, I was able to set the actual version of the JDK to use by Gradle
by editing the project’s
19 (or the latest):
The next issues are related to machine learning issues like numerical computations, datasets, etc.
TensorFlow Model not Learning due to Unshuffled Training Dataset
When I was training a TensorFlow binary classification CNN model I had to change/tune some code for training and measuring.
The peculiarity of this problem is that training data has to be shuffled so the training doesn’t get biased due to the way data is sorted in the file system. If training takes place with default-sorted disk data then the model won’t learn, and that was the problem I had.
The API is the following:
list_files | tf.data.Dataset | TensorFlow v2.12.0
I was explicitly passing
shuffle=False since this was necessary for getting
metrics correctly (after training). That is, the same order of files was
required to map them for computing the model metrics on the datasets.
Then, I put a model to train with this new in-house API I was developing, but it wasn’t learning 😬 (something you can check in the training history logs/plots).
After checking everything, I realized the only meaningful change I had done was
that argument when loading the dataset. So, I proceeded to set it to
(which is the default), and the training behavior was fixed.
This might possible since the dataset will provide a default pattern according to how it’s stored, and this will probably introduce bias, and the model might get locked in a local minimum as per what I was reading about how the Stochastic Gradient Descent (SGD) works.
Always make sure to shuffle data before training to avoid biases.