Intriguing PLC troubleshooting job

Had the privilege of taking up a very interesting (should I say challenging) troubleshooting job that involved PLC control recently.

As I do not have the permission to reproduce any specific information relating to pictures or system architecture, I should just share the takeaways and the summary of the job.

Essentially the Power Management System (PMS) system is not receiving any data from a particular Diesel Generator (DG) via the DG's PLC. The PMS therefore does not permit any remote operations on the DG.

The PLC obtains DG data from two sources: The DG controller (Deif) and the DG ACB (Schneider).

What could be defective?

1. PLC (whose program is locked, so there's limitation to how much can be done here)

2. DG Controller

3. ACB

4. Cabling involving two of the sub-systems

While the conclusion looks near superficially (how difficult can it be when we only have 4 sub-systems to investigate and/or replace), truth could not be further away from that.

[2], [3], [4] were changed. But problem remained.

We managed to find access to the PLC program. So [1] was changed as well.

Interestingly, the problem was only partly solved after the PLC was changed. It was only fully resolved after both the PLC and DG Controller were changed.

So what is the takeaway for us:

Never take any assumption as conclusion on what could be defective or not. In this case, we were so sure that the controller was not responsible. But thankfully, with an open-minded customer who suggested that we exhaust every permutation, the problem was resolved.

Was indeed an experience of very high pedagogic value.