Turn up the Logs!

Whenever there is a problem in Maximo and you describe it to one of your co-workers the first thing they say to you is “what do the logs say?” Logs are the first place to look when you have a problem. There are some small issues with logs in that they are all over the place and some you will need to activate to access what can be logged. When you install Maximo (with WebSphere as it’s middleware) a couple of directories will become very familiar to you. They are the SMP and WebSphere directories. The logs are in there. Deep down in there. So, lets talk about making life a little bit easier. When you set up any environment you will go through some initial setup that we would call the foundational setup. Setting up your foundation is critical for managing and maintaining an environment that runs well and does not frustrate the hell out of you because everything is all over the place. Logs are almost always overlooked, and you end up having to go to all the standard locations to dig up the information that you are looking for when things go wrong. In this article I will show you how to set up your logs so that you can retrieve them in a more polished, sophisticated way that will exemplify how a well-built environment runs.

While researching information for this blog article I came across the usual suspects and for the Maximo users out there you know what I’m talking about. Log locations, log file sizes, amount/level of what will be logged. How Maximo uses the Log4j logger. Those are all great and form the foundations of how the application can be checked. I get the most info from the WebSphere logs though. Specifically, the SystemOut.log and the SystemErr.log. These are WebSphere’s logs that are writing information about the application that it is hosting (in our case Maximo).

The first thing you would consider when setting up your logs is where they are located.  The default path is going to be something like this (C:\IBM\WebSphere\AppServer\profiles\ctgAppSrv01\logs\MXServer). I’ve gotten quite used to that being the log location, but you may want to make life a bit easier and put it into a directory like C:\WebSphere_logs. The path (location) of the SystemOut.log and SystemErr.log can be changed. If you go to the servers and click on your server there will be a link on a right-hand navigation that will be called Logging and tracing under the Troubleshooting section.

When that page opens you will see a link called JVM logs. When clicked this will be a page most Maximo workers will know, and it gives you some very good options as to how you want your WebSphere logs set up.  As you can see from my screen capture below the log files were increased from the 1 MB standard to 5 MB, because even in environments that have very little traffic a 1 MB log will fill up quickly (like get your timer out quickly).

What size your log files should be is going to be a call you’ll have to make.  Take into consideration things like your server environment, your usage of cron tasks, automation scripts, escalations, and integrations.  These are all likely to be written to the SystemOut.log file if you are running them or increasing log levels to find out what they are doing.  You can also set how many “rolling” logs you want to keep under the Maximum Number of Historical Log Files field.  On this same page you can click on the Runtime tab and you will have the option to change the path the SystemOut and SystemErr files are written to.  This is where you can change the default repository for your logs to something that is not as buried as the original location.

Now, let’s go a bit deeper into what WebSphere can do for us when logging application information.  When the standard information that the SystemOut and SystemErr is not showing anything from the errors you are receiving it may be time to “turn up” logging. As I mentioned before Maximo is an application hosted by WebSphere, and as any developer knows tracing an application will provide step by step information as to what the application is doing. WebSphere allows you to turn on tracing of its applications. This is not on by default and you will want to turn it off once you are finished due to the amount of resources that will get chewed up while tracing an application. On the same server page (Application servers > MXServer > MXserver) where we went into to access the JVM logs there is a link for Change Log Detail Levels.  Here we can “turn up” tracing. This will allow for deeper diving into the actual applications that WebSphere is running.  The default setting is *=info.  This is going to be plenty good enough for 90% of your troubleshooting. Now, let’s say strange problems are going on with your hosted applications (Maximo) you may want to “trace” the applications inputs and outputs and as you know that’s all programs are. An interface for input and it spits an output as a result of what it was given.  If you increase the level to *=fine, *=finer, or *=finest you will now get a trace.log to pop up in that good old default directory. What this will relate to you will be a trace of what the application is doing.  Depending on the components you add the more “granular” a set of detail you can log.

 

IBM has a very good article on their Knowledge Center entitled Log Level settings which details levels that can be traced.  (https://www.ibm.com/support/knowledgecenter/SSEQTP_9.0.5/com.ibm.websphere.base.doc/ae/utrb_loglevel.html).  You are not going to want to leave your WebSphere system in this state because this of course will chew up resources at a much greater rate than just a few rolling logs that are written to every minute (the SystemErr.log doesn’t necessarily write every minute.)

Let’s take into account when systems need to run lean and clean and have to push the system’s resources (otherwise known as processor and available memory) to their maximum potential.  So, you’re hosting Maximo on your raspberry pi and you know you will have 200 concurrent connections at any given time editing work orders, creating purchase requisitions, firing automation scripts from custom built apps and you know that 1.2GHz quad core processor is going to be working hard, you may benefit from altering your logging system. WebSphere has the ability to log in HPEL mode.  High Performance Extensibility Logs are the exact same thing as your SystemOut, SystemErr, and trace logs, but they are not written to a nice easily accessible text file with the .log extension. These logs are written in binary and I am not even geeky enough to read that. I’ll need a translator to read binary. Lucky for me the HPEL logs can be accessed with a tool called logViewer.bat (or in the case of your raspberry pi logViewer.sh). This tool can translate the binary and bring back the information to you in either a shell scripting tool or command prompt as well as give you the ability to take the retrieved data and have a text file created out of it. Is it hardcore? The answer is absolutely, but as an old saying goes extreme situations require extreme measures.

When you go to set this up you will see a statement from WebSphere saying “Advised for most installations.” I would venture to say this is not recommended for most installations. Your choice is to have the system use some memory and processor threads to write to a text file that you can set to roll over and be readily available to check out or use a command line tool to extract the binary data and present it to you or create the text file on demand. I’m going to go with having a nicer system that has more memory and bigger processor so that I can easily read my log file.

In a final parting statement, it is a best practice to return the logs back to the state you found them in. If you have “turned up” logging, “turn it down”. It’s just going to convolute the next person that has to see a lot of excess info when they are opening the logs to troubleshoot. This topic can be extensive, and I encourage everyone to read up on what logs you may need to use in your endeavors. I could keep rambling and rambling, but my goal is to pique your interest not put you to sleep.  Stay tuned for more blog articles and go get those problems fixed by seeing what it says in the logs!

Related posts

2 comments

  • Biplab Das Choudhury

    Good informative article but I would not go into trace logging unless the issue is really not found with Maximo logging like tls version issue or sso.
    Maximo Logging application is powerful enough to triage any Maximo related issue.
    Having said that I think the information you have shared here is gold and cannot be found easily.
    Enjoyed the read!

  • Anonymous

    text

Shane

Maximo Analyst

As a recent graduate in computer science and cybersecurity with a CompTIA Sec+ certification, I thrive on learning and embracing emergent challenges head-on. Though new to Maximo, I approach the day to day with determination and enthusiasm. I have been known to be competitive while also being a strong team player. Beyond the realm of technology, I cherish spending time with loved ones, playing video games, doing exercise, and cooking. I am grateful to be where I am and around so many amazing people.

MAS 8

Time to Benefit
Lower Costs
Scalability and Integration
Continuous Upgrades
Easy of Use

Visual Inspection

Accelerate Defect Detection
Flexibility to Train and Deploy
Increase Uptime 24/7 Real Time Monitoring
Better Worker Safety

Monitor

Enterprise Level Data Aggregation
Scaled Operations
AI Deductions
Advanced Root Cause Analysis

Predict

Condition Based Maintenance
Lower Costs
Better Asset Availability
Reduced Risk

Manage

Comprehensive View of Assets
Understand Resource Attributes, Configuration and Relationships
Drill Down View

Health

Improve Reliability
Performance Insight
Better Planning
Clean Data
Optimization – Data, Costs, Risks

Mobility

Boost Productivity
Safer Workplace
Reduce Knowledge Gaps
Avoid Downtime

Digital Twin

Integrate with IoT for Intelligent Asset Management
Measure Health and Performance
Meet Sustainability Goals
Write Your Digital Story – Learn from Data

Dexter

Maximo Analyst

Dexter began his career in the warehouse field. Developing his skills in inventory control, accounts payable, shipping, and receiving, and procurement. While working at Apex he was learning the Maximo system in and out. Eventually learning it so well he taught everyone how to use it. Eventually becoming a system administrator for the Apex site.

A fun fact about Dexter is that he was an Elvis impersonator for 24 years. He enjoys singing, fishing, golf, and playing pool. He's an avid movie buff, most likely you'll find him watching the latest movie.

Chip

Client Relations Specialist

Chip is new to Maximo but has worked for various software companies through his career in both a sales and client relations role. In his spare time, Chip likes to golf, travel and work on projects around the house, especially those with an artistic touch. He is also up for any type of competition whether that be playing cards, board games or recreational sports.

Sarah

Maximo Analyst

Sarah is a recent graduate starting her asset management career. She counts fast learning and determination as key skills that she’s applying to become an Maximo pro! Sarah’s background in IT Services and studies in computer science have strengthened her natural problem-solving skills and desire to help others through technology. In her free time, Sarah enjoys baking, sewing, reading comic books, and building her own video game.

Derrick

Maximo Analyst

Derrick is someone who lives for mental exercise. New challenges spin his problem solving wheels. Defeat is not an option and challenges make him smile. His catch phrase is “I’m an Engineer”. Derrick takes pride in his work. He is a highly self-driven talent when it comes to his interests and programming couldn’t be any higher on this list.

Outside of the office Derricks interests includes cars, motorcycles, and anything generally fast and loud. He stays in the loop on latest in tech, loves gaming and most of all is a die-hard anime fan. Outside of bike night, you are most likely to catch him relaxing in the glow of lights from his PC.

Xavier Galarza

Maximo Analyst

I am a recent graduate who is motivated to learn and develop new skills. I like to challenge myself every day. I want to learn all about Maximo in order to teach others how effective Maximo can be for them. I have a good understanding of the development side and work well with problem solving. I am grateful for the position I am in, I love surrounding myself with great people that are more knowledgeable than me who will help me grow as a person. In my free time I love to play videos games, watch/ play sports and spend time with family. I am a competitive person and believe I am pretty good at all sports.

My other passion is to stream video games I play and connect with others online, I love showing others how to play a certain game or even show them a skill didn’t know existed. I love entertaining others and creating content for others to watch and enjoy.

Courtney LeBlanc

Business Analyst

Courtney is a polished Maximo consultant with over 10 years working in the industry. She has worn many hats including QA Specialist, Trainer, Functional Lead, Project Lead and Business Analyst. A point of great professional pride for Courtney is helping clients meet their needs and realize the power of their investment. Being a part of the process that helps clients learn how to use their system and get the most out of it is a great feeling at the end of the day.

Bob Richardson

Director of Managed Services and Senior Analyst

Bob has led more than 50 (but less than ∞) Maximo projects with top notch client satisfaction – so he must be doing something right. He is a seasoned veteran who heads our operational & managed services team. He is instrumental in developing many of our solutions and holds Maximo certifications in both functional and technical disciplines up to version 7.

Bob might or might not like squirrels. He enjoys traveling with his wife and outdoor activities. He has also been known to spend long hours in his garage contemplating life.

Kelly Nimmo

Director of Products and Senior Analyst

Kelly heads up our product development team and is a senior analyst with more than 11 years EAM/Maximo experience. She is the architect behind our flagship product offering MxMobile. She holds Maximo certifications in both functional and technical disciplines up to Maximo version 7.

In her free time if you don't find her on her boat, fishing and enjoying the beautiful Florida outdoors, then she's probably on the soccer field teaching her nieces and nephews how to do the around the world.

Kim Walter

COO

Kim began her career in Investment Banking in the Biotech and Tech sectors. After leaving banking, she focused on executive strategic development evolving her business building skills. Co-founding A3J Group was a natural fit with the idea of creating a fierce ninja style unit that outperforms all others. In her free time, she is training for her next eventing competition or studying to become a Master of Wine or practicing her tornado kick. She could be with her family hiking or surfing or cooking. But she’s definitely not sitting around doing nothing.

Alex Walter

CIO

Alex brings 18 years of Maximo/EAM consulting experience in Life Sciences, Oil & Gas, Water/Waste Management, Government facilities, and much more. He has been awarded the honor of IBM Champion from 2018-2021, and is the brainchild behind our revolutionary mobile solution MxMobile.

In his free time he buys camping gear, enjoys a good circular saw and spends time with his family hiking, surfing and being outdoors.

Enhancement

• Configuration
• Integration & data loading
• Report improvement
• Screen view & workflow changes

Sustainability

• Report
• Troubleshoot
• Learn
• Teach
• Lead

Change Management

• Advanced systematic improvements
• Tailored procedure implementation
• Script-, report-, and application-level source and document control through Git repository software
• Training on how to document and deploy changes

Monitoring

• Active monitoring of your IBM Maximo system’s heartbeat
• Triggered notifications and support tickets when Maximo is not responding
• Automated Escalations based on the health of various critical background processes.
• Periodic and quarterly scheduled system health checks

System Support & Trouble Shooting

•On Demand Support
•Advanced Troubleshooting
•Access to A3J Group’s Maximo Support Environment to submit trouble tickets
•Directly report bugs, request fixes and enhancements

System Patching & Environment Refreshes

• Tentative, bi-annual feature pack releases from IBM
• Minor patches (defined by the last number in the version model)
,• Quarterly lower environment refreshes

Admin

MxAsk

Create new self-service requests in a wizard-style format, and view the details of their open service requests.

MxApprove

Accept, reject, and reassign assignments, review assigned records, and view your workflow inbox.

MxMobile

A suite of mobile applications designed to integrate with IBM Maximo Asset Management. These apps modernize workflow, expedite inventory processes, reduce time spent on data entry, and more.

Workflow

MxMeter

View and enter meter readings, view trends in graphical format, and create readings while in offline mode.

MxAsset

Scan barcodes and view maintenance facts. Create meter readings, work orders or service requests. Toggle asset downtime.

MxWork

Create new work orders on the go, view existing work orders and attachments, and capture fingertip electronic signatures.
0