Data Logger Demo has a sampling offset
응답
15. 7. 28 오후 1:00
Dear XDK-Community,
 
I have found an offset in the data sampling rate using DemoDataLogger Code (v1). All the sensor data is sampled with the sampling rate 50Hz. The logging time is 30min+/-1sec.
 
Theoretically in total 90,000 (50Hz * 60sec*30min) Sampling-Data shall be stored in the SD card. However only 72 088 Sampling-Data is saved.  So instead of a sampling rate 50Hz, the real sampling rate is about 40Hz.

Holma has the same issue (see Thread: “Data Logger Demo Application“):

Dir Philipp,

the data logger application works fine now.
Only the values of the timestamp column seems to be not very acurat.

I have made a logfile (all sensors on; 50 Hz sampling) over exactly on hour.

Last timestamp value = 3350120 ms; expected about 3600000 ms; Difference =  249880 ms (6,9 %)

I have also made a logfile (all sensors on; 100 Hz sampling) over exactly on hour.

Last timestamp value = 2503668 ms; expected about 3600000 ms; Difference =  1096332 ms (30,4 %)

Have You any explination for the differences ?

Kind Regards
  Manfred


Now the question is, does it lies on the slow writing speed into SD card or is it due to the accuracy of system clock? 
 

Best regards
Stefan
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 7. 29 오전 7:11 as a reply to Xinyi Yang.
Hi Stefan,

first of all, thanks for taking the time to give an accurate description and pointing back to Manfreds post on the same topic.

When Manfred posted it, I forwarded all the info to our second level, but I haven't heard back from them yet. To ensure that we're getting a info on this, I'll write them again now. 

Meanwhile, I don't think the system clock is what's causing this problem. 
I'm actually having an idea what could be the problem here, but let's verify something first: Which sensor are you trying to read in the setup you are reffering to?

- Florian
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 7. 29 오후 2:46 as a reply to Florian Harr.
Hello Florian,

we are reading all the inertial sensors and environment sensor.

best regards,

Xinyi
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 7. 29 오후 5:12 as a reply to Xinyi Yang.
Stefan,

could you please share your logger.ini file share with us?

- Florian
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 7. 30 오전 8:13 as a reply to Florian Harr.
Hi Florian,

here you are.

------------------------------------------------ logger.ini ------------------------------------------------------------------
; Important note: The shortest saving time on the SD-Card from the Sensor is 2ms per Sensor.
; So if you use all 6 sensors, config. all of them to 1kHz (1ms), it will save each sensor data only every 12ms.
; Reason is the slow I⊃2;C bus connection, where the Sensors are connected.
; The Sensors are able to sample every 1ms (1kHz), but the request and getting the data from each sensor takes 2ms.
; 1kHz could maybe be realized through a SPI implementation.
; Savingtime = 2ms * Sensor (IC)

; Instruction
; enabled:    Activates the Sensor Block (IC). Each sensor block (IC) can have 1 up to 6 sensor signals
; range:     The range represents the physical value for example 2*g (earth acceleration).
;         The step amount is the same, but if the total range is bigger, then each physical step is also bigger,
;         for that the resolution will shrink.
; bandwidth:    It means the highest possible signal frequency --> according that, the filter will be set (preprocessing)
; sampling_rate:Is reciprocally tot he sample time
; oversampling: It will increase the sampling time (multiply) to decrease the quantization noise
; filter_coefficient: The more coefficient the filter has, the smaller is the fall time of the filter .
;         Many coefficients = abrupt filtering of Signals; Less coefficients = fluent filtering of the Signals.
; integration_time: The Integration time is the time of integrating the measured value.
;         The higher the integration time is, the higher is the resolution. This means,
;         if the integration time is very high then it will detect low light emission very good.
;         If there is very bright light, then the integration time should be set lower to prevent saturation.
;         Note: if the integration time is set to 800ms, then the sampling time is also limited to the 800ms --> maybe 1s !
;         (for more details see: http://datasheets.maximintegrated.com/en/ds/MAX44009.pdf or https://en.wikipedia.org/wiki/Integrating_ADC)


[general]
; Specify filename in sprintf format
filename=log_%04i.csv
; Output file format: [csv, json, custom]
fileformat=csv
; Output data format: [raw, unit]
dataformat=unit

[date]
; Year in that XDK will be powered on: [2015,2017-2019] All years without all year=2016+(n*4) with n=1,2,3,... since February with 29 days is not implemented
year=2015
; Month in that XDK will be powered on: [1-12]
month=7
; Day in that XDK will be powered on: [1-28,30,31]
day=22
; Hour in that XDK will be powered on: [0-23]
hour=22
; Minute in that XDK will be powered on: [0-59]
minute=0
; second in that XDK will be powered on: [0-59]
second=0
; Fastes sampling time (fastestSamplingTime=1/fastestSamplingRate) in ms [1-1000]ms
fastestSamplingTime=20

[bma280]
; Sensor enabled? [0, 1]
enabled=1
; Sensor Sampling Rate in Hz: [0.0001 - 1000]
sampling_rate=50
; Sensor Range in +- g: [2, 4, 8, 16] --> Set to 2 since it doesn't comes over 2*g
range=2
; Sensor Bandwidth in Hz (or unfiltered): [7.81, 15.63, 31.25, 62.5, 125, 250, 500, unfiltered]
bandwidth=31.25

[bmg160]
; Sensor enabled? [0, 1]
enabled=1
; Sensor Sampling Rate in Hz: [0.0001 - 1000]
sampling_rate=50
; Sensor Bandwidth (highest Signalfrequency) in Hz (or unfiltered): [32, 64, 12, 23, 47, 116, 230, unfiltered]
bandwidth=23

[bmi160]
; Sensor enabled? [0, 1]
enabled=1
; Sensor Sampling Rate in Hz: [0.0001 - 1000]
sampling_rate=50
; Accelerator Sensor Range in +- g: [2, 4, 8, 16] -> --> Set to 2 since it doesn't comes over 2*g (according the total range, the stepssize will be set --> Bigger Range means larger steps and lower resolustion)
range=2
; Accelerator Sensor Bandwidth in Hz (or unfiltered): [7.81, 15.63, 31.25, 62.5, 125, 250, 500, unfiltered]
bandwidth_accel=31.25
; Gyroscope Sensor Bandwidth in Hz (or unfiltered): [32, 64, 12, 23, 47, 116, 230, unfiltered]
bandwidth_gyro=23

[bmm150]
; Sensor enabled? [0, 1]
enabled=1
; Sensor Sampling Rate in Hz: [0.0001 - 1000]
sampling_rate=50
; Sensor Datarate in Hz (or unfiltered): [10, 2, 6, 8, 15, 20, 25, unfiltered]
data_rate=25

[bme280]
; Sensor enabled? [0, 1]
enabled=1
; Sensor Sampling Rate in Hz: [0.0001 - 1000]
sampling_rate=50
; Sensor Oversampling  (or off): [off, 1, 2, 4, 8, 16] --> High Oversampling decreases quantization noise but maybe increase signal noise.
oversampling=16
; Sensor Filter Coefficient  (or off): [2, 4, 8, 16, off] --> High
filter_coefficient=4

[MAX44009]
; Sensor enabled? [0, 1]
enabled=1
; Sensor Sampling Rate in Hz: [0.0001 - 1000]
sampling_rate=50
;Sensor Integration Time in ms [800, 400, 200, 100, 50, 25, 12.5, 6.5 ]
integration_time=50

------------------------------------ end -------------------------------------------------------

Best regards,

Xinyi Yang



 
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 7. 31 오전 4:54 as a reply to Xinyi Yang.
Hi Xinyi,

thanks for your input on this.

Give me a few days to follow up here as I would like to test it again myself with a few different values.

- Florian
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 8. 5 오전 4:13 as a reply to Florian Harr.
Hi Xinyi,

I did a few test, here are my findings:

1. Test: 60 Minutes runtime, 6 Sensors all at 1Hz (1000ms) - Full Bandwidth

#Number of entries: 3690
Last timestamp: 3689000 (1,02 hrs)

#Number of expected entries: 1Hz * (60seconds á 60minutes) = 3600
-> Conclusion: 90 Entries too much. I file that under human inaccuracy when starting and stopping the XDK.

2. Test: 60 Minutes runtime, 6 Sensors all at 1000Hz (1KHz) - Full Bandwidth. Min save time = 12ms

#Number of entries: 211 344
Last timestamp: 2536116 (0,70 hrs)

#Number of expected entries: (1000Hz)/83.333Hz * (60seconds á 60minutes) = 299 880
-> Clear offset

Now there's a clear offset and I did more test inbetween 1Hz and 1000Hz. These showed that with higher sampling rates, the offset got greater. Before we continue here, let me pick out a few things that are important to consider here:

The logger.ini clearly says:
; Important note: The shortest saving time on the SD-Card from the Sensor is 2ms per Sensor. 
; So if you use all 6 sensors, config. all of them to 1kHz (1ms), it will save each sensor data only every 12ms. 
; Reason is the slow I⊃2;C bus connection, where the Sensors are connected. 
; The Sensors are able to sample every 1ms (1kHz), but the request and getting the data from each sensor takes 2ms. 
; 1kHz could maybe be realized through a SPI implementation. 
; Savingtime = 2ms * Sensor (IC) 
So after studying the implementation, it seems that this is a factor that's affecting us here.

Furhtermore, if you look into the download threat for the  DataLogger Demo you'll find a detailed warning that's picking out exactly the mentioned problem here:
  • Under certain circumstances, sensor data is not sampled often enough, resulting on repeatedly logged values.
  • Rarely, gaps in the timestamps larger than your sampling interval can be observed.
  • Some SD cards seem to create problems and trigger the "internal error" LED of the data logger
Now I understand that this is a huge problem, but as you can see the BCDS team knows about these problems and is working hard on getting these things fixed.

In the meantime I hope this helps you to see the bug is being acknowledge here and in the thread you were quoting.

If there's anything else we can help with, please let us know!

We're more than happy to help where we can.

- Florian
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 8. 12 오후 1:18 as a reply to Florian Harr.
Hello Florian,

thank you for the explanation. However I don't quite get why the limit of saving data to SD card could affect our test result so heavily.
We are using 50Hz sampling rate, which means the time interval would be 20 ms.
The saving time needed for 6 sensors are 12 ms, which is much smaller than than the time interval. I will try to reduce the number of sensors sampled and check the result once again.
 
best regards,

Xinyi
 
0 (0 투표)
Data Logger Demo has a sampling offset
응답
15. 8. 12 오후 1:33 as a reply to Xinyi Yang.
Xinyi,

again, please see my explanations above. It's not only the I2C -> SD Card connection but many other things as well, which is why BCDS gave out the warning I was quoting in my post above.


- Florian
0 (0 투표)