Hi David,

Thanks for all the data. That provides some additional clues but we're still not able to figure out what the problem is.

The 180us and 270us time slices are correct and make perfect sense. With only the diagnostic program running (with the system Thread always also running) the time should be 2x90us=180us. When your homing Thread is also running the time should be 3x90=270us.

The 360us time is incorrect and should not occur. I realize now that there is a flaw in the diagnostic program. It basically computes elapsed time based on the number of 90us servo ticks plus an offset from the last servo tick. But if a servo tick is not 90us then the time will be incorrect. I've attached a new diagnostic that measures the time as before and also using a raw 50MHz timer that should give a correct result regardless of if a servo tick is delayed. Both results should agree if the servo ticks are really 90us.

I'm not sure how interested you are but here is my current theory on what is occurring (but I don't yet understand why). Its a somewhat complicated matter but basically KFLOP's DSP scans the Axis Channel's configurations and determines which of 32 registers in the two SnapAmps need to be read. It then forms a 32-bit word (with one bit for each register) and writes the word to KFLOP's FPGA. KFLOP's FPGA then takes over and reads all those registers from the SnapAmps through the 16 wire ribbon communication cable. This could be up to 512 bits and require 10~20us. The results are stored into a FIFO in KFLOP's FPGA. KFLOP's DSP then reads the data from the FIFO. Normally by the time the data is attempted to be read it should already be available. If it is not available it loops up to 2048 times (~1ms) until it is or times out. I believe for some reason the data is not available and this stalls processing for ~1ms (or multiples of this depending on how many timeouts occur). This of course greatly disrupts the 90us servo samples.

The communication between KFLOP and SnapAmp is timing deterministic (it is basically like reading memory). Regardless of noise or communication errors the data should be read in the same amount of time (it could be corrupted data but should never hold off putting something in the FIFO to be available to be read by KFLOP's DSP).

One scenario that might fit could be that the 32-bit word that determines which register are to be read might be corrupted when it is written locally to KFLOP's FPGA. If some bits were altered from 1 to 0 then those registers would not be read and placed in the FIFO as expected causing a timeout. However I can't think of any reason why a noise problem like this would be suddenly switched on by some event. Why would millions of servo ticks work flawlessly and then suddenly failures occur randomly every few thousand servo ticks? I suppose Temperature could do something like that?

Odd things about your configuration is that you have 2 SnapAmps plus a Kanalog. Is it possible to run your system without Kanalog connected to see if that is somehow related and triggereing a bug in our code?

Another Odd thing is that you are using KFLOP Axis Channels 0,2,4,6 for some reason. Normally Axis channels 0,1,2,3 would be used. That should not make a difference but you might try to see if it does. Also you might set unused Axis channels to Input Mode "No Input" (KFLOP reads and maintains encoder positions for all Axis channels set to encoder input mode regardless of whether the channels are used or enabled).

There is an example Measure IRQ.c that measures the Servo Calculation Time for all the Axes and other operations that occur every 90us servo sample (which should be significantly less than 90us). You might run that before you observe the problem and after to see if there is a difference. If there is, that might be a clue. Note: no other User Threads should be running to get a valid measurement.

I suspect this is a software bug or noise issue. But we could send a loaner set of boards to try to see if this is some hardware problem with the boards. Contact our support if you would like to do this.

Are the 4 board stacked together with standoffs and short cables? Are there any earth ground connections to the boards?

Sorry for more questions than answers.

Regards