Topic: Regularly missing data in custom server function on GW2000

[ocean]

Hey,

for about a week I have activated the custom server function on my GW2000 gateway now and it mostly works without issues at interval 60.

However, I have noticed that in the .csv file created by a .php script on my server, there is very regularly a minute missing about every hour.

Because it happens so regularly at every single hour, I wonder what could be the cause of it? It surely can't be a connectivity issue, as that would happen more randomly and for longer I guess.

I created an Excel makro to quickly mark the cell below a missing entry, in the attachments I put an easy to access pdf sample of what I mean.

Does anyone know a solution to this? Or what causes it?

Thank you so much for your help. I appreciate it a lot.

[Rover1822]

Are you also sending to Ecowwitt.net , and do the drop outs match?
So it is in effect missing a one minute interval?
And this happens at the hour mark?

[Autofill]

Anecdotally, I believe I've noticed this too with my custom server settings for my GW2000.

I'm using an integration called ecowitt2mqtt for my home assistant add-on and have it set for 16 second updates, I'll have to do the math to confirm this but it does skip updates.

There was another chat here on the forum, other people noticing this. I think the bottom line is is just report this to Ecowitt with your numbers and then they'll fix it on the next FW update I'm sure.

[olicat]

Hi!

there is very regularly a minute missing about every hour.

In fact, the interval of 60 does not mean that the console sends data every 60 seconds, but that it waits 60 seconds after a transmission.

The transmission itself also takes approx. 1 second, as can be seen from the offset of approx. 1 seconds for the individual transmissions. Or the console waits an additional second before or after a transfer.
These missing 1 or 2 seconds had already given me a lot of time for debugging my own programmes - I was looking for the fault - the loss of the 2 seconds - with me.
Apparently, however, it is really the console itself that is doing this!
For reasons that are unclear to me. Because at least the GW2000 could work with several threads and do the data transfer in parallel. Or the data transmission could be interrupt-controlled by a timer.


Because of these "rolling seconds", a transmission is missing once an hour (or 2 if there is missing 2 seconds):

Code: [Select]

#. hour min second
1   00  00  00
2   00  01  01
4   00  02  02
5   00  03  03
6   00  04  04 
7   00  05  05
8   00  06  06
9   00  07  07
10  00  08  08
11  00  09  09
12  00  10  10
13  00  11  11
14  00  12  12
15  00  13  13
16  00  14  14
17  00  15  15
18  00  16  16
19  00  17  17
20  00  18  18
11  00  19  19
12  00  20  20
23  00  21  21
24  00  22  22
25  00  23  23
26  00  24  24
27  00  25  25
28  00  26  26
29  00  27  27
30  00  28  28
31  00  29  29
32  00  31  30
33  00  32  31
34  00  33  32
35  00  34  33
36  00  35  34
37  00  36  35
38  00  37  36
39  00  38  37
40  00  39  38
41  00  40  39
42  00  41  40
43  00  42  41
44  00  43  42
45  00  44  43
46  00  45  44
47  00  46  45
48  00  47  46
49  00  48  47
50  00  49  48
51  00  50  49
52  00  51  50
53  00  52  51
54  00  53  52
55  00  54  53
56  00  55  54
57  00  56  55
58  00  57  56
59  00  58  57
60  00  59  58
61  01  00  59
--- missing minute ---
62  01  02  00

I would also like to see a change in broadcasting behaviour. Away from the pause of the entered interval length to an actual interval.
But that's just the way things are now.

By the way, this behaviour affects all Ecowitt consoles.
With the HP2551C and the HP3500, however, it is much worse, because at least with very many sensors the entered interval is never even approximately adhered to. It also happens that instead of the set 60 seconds, only every 90 seconds is sent.

Regards, Oliver

[Gyvate]

With the HP2551C and the HP3500, however, it is much worse, because at least with very many sensors the entered interval is never even approximately adhered to.

The reason for is, as per Ecowitt, that the WiFi modem of the HP2551 and HP350x consoles needs a lot of processing power, whereas the GW1x00 and GW2000 consoles have a WiFi modem SoC with its own processor. Therefore the latter can handle more sensors and shorter posting intervals better than the HP or WS (2320E, 2910) consoles.
The "dropping" of a post after a certain time, depending on the posting interval chosen, I have also observed, and to me it looks that there is an algorithm behind which (indirectly) creates the drop when multiples of the real-world interval (which is usually different from the chosen one, and be it by one second: 17 instead of 16 seconds or 36 instead of 30) do no longer fit into the left time to complete a full minute or only a small portion of it. Would need to check a lot of different posting intervalls over let's say 10-15 minutes and look for the dropping point and compare amongst the different intervals.

[ocean]

Thank you very much for all your answers so far 

Are you also sending to Ecowwitt.net , and do the drop outs match?
So it is in effect missing a one minute interval?
And this happens at the hour mark?

Correct, I am also sending to ecowitt.net simultaneously, also at a 60s interval.
However, it's much harder to actually to prove drop outs there, as all the data gets saved "only" in a 5min interval on their cloud.

Hi!

there is very regularly a minute missing about every hour.

In fact, the interval of 60 does not mean that the console sends data every 60 seconds, but that it waits 60 seconds after a transmission.

The transmission itself also takes approx. 1 second, as can be seen from the offset of approx. 1 seconds for the individual transmissions. Or the console waits an additional second before or after a transfer.
These missing 1 or 2 seconds had already given me a lot of time for debugging my own programmes - I was looking for the fault - the loss of the 2 seconds - with me.
Apparently, however, it is really the console itself that is doing this!
For reasons that are unclear to me. Because at least the GW2000 could work with several threads and do the data transfer in parallel. Or the data transmission could be interrupt-controlled by a timer.


Because of these "rolling seconds", a transmission is missing once an hour (or 2 if there is missing 2 seconds):

Code: [Select]

#. hour min second
1   00  00  00
2   00  01  01
4   00  02  02
5   00  03  03
6   00  04  04 
7   00  05  05
8   00  06  06
9   00  07  07
10  00  08  08
11  00  09  09
12  00  10  10
13  00  11  11
14  00  12  12
15  00  13  13
16  00  14  14
17  00  15  15
18  00  16  16
19  00  17  17
20  00  18  18
11  00  19  19
12  00  20  20
23  00  21  21
24  00  22  22
25  00  23  23
26  00  24  24
27  00  25  25
28  00  26  26
29  00  27  27
30  00  28  28
31  00  29  29
32  00  31  30
33  00  32  31
34  00  33  32
35  00  34  33
36  00  35  34
37  00  36  35
38  00  37  36
39  00  38  37
40  00  39  38
41  00  40  39
42  00  41  40
43  00  42  41
44  00  43  42
45  00  44  43
46  00  45  44
47  00  46  45
48  00  47  46
49  00  48  47
50  00  49  48
51  00  50  49
52  00  51  50
53  00  52  51
54  00  53  52
55  00  54  53
56  00  55  54
57  00  56  55
58  00  57  56
59  00  58  57
60  00  59  58
61  01  00  59
--- missing minute ---
62  01  02  00

I would also like to see a change in broadcasting behaviour. Away from the pause of the entered interval length to an actual interval.
But that's just the way things are now.

By the way, this behaviour affects all Ecowitt consoles.
With the HP2551C and the HP3500, however, it is much worse, because at least with very many sensors the entered interval is never even approximately adhered to. It also happens that instead of the set 60 seconds, only every 90 seconds is sent.

Regards, Oliver

Just like you, for the better part of the whole week I spent debugging my own scripts, going through the console settings over and over, checking my server etc. to find if I did something wrong. But it truly seems to be due to settings we can't access. Hopefully Ecowitt will do something about it at some point.

Although, let's be honest, the vast majority of people using Ecowitts hardware won't ever notice these little inconsistencies, so I doubt it would be high on their priority list to fix.

Especially as there seems to be no actual data lost, as far as I can tell from my own observations so far?

It's just slightly annoying and activates my inner Monk 

[hiljo]

See also my post about this that is somewhat related to your question: https://www.wxforum.net/index.php?topic=42587.0

[S1m0n3]

I don't know if it's related in any way, but I've had it happen to me already twice, that I have a different maximum wind gust on ecowitt.net and my custom server, as if the custom server is losing some records on the way. It could be a server problem and not related to the gateway.

[Gyvate]

it's maybe not really losing. It may be only a different set of data which will be accumulated and either the average or maximum value will be posted. When it posts to several services, the custom server comes last. Depending on the payload of the other postings some more time will have passed => a different interval and resulting values will/could be considered.

[ocean]

As a quick follow up and to anyone in the future having the same issue looking for a fix:

The only thing that as of December 2022 really works to fix the common and regular "drop outs" is to set your custom server interval at 59 seconds.
It's the only work around that I actually found works long term.