XDK InfluxDB and Grafana
응답
18. 1. 17 오후 5:52

Hello, i have done today a lot of trials with posting XDK sensor data into a influxdb and showing the data on a Grafana Dashboard. Finally i have a working solution - i can post the XDK data with my SmartSensor Application - but also directly with the XDK.

Biggest problem - i got no connection on Port 8086 if i compiled my code with Workench 3.x.
Always got error -111 - only if i used port 80 data was send correctly.

Then i switched down to version 1.7 WB - same code - no problem. All ports working.

What could be the reason or problem that port 8086 or others than 80 not working on WB3.x Systems? This was also a problem from Mr. Koch testing my game PengoBrain with different ports.

I will show and present my solution on my Alexa news feed channel and blog soon.
So every user will be able to see his sensor data on a nice Grafana Dashboard.
 

Greetings Achim

 

+1 (1 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 17 오후 5:27 as a reply to Achim Kern.

Hello Achim,

good to hear again from you, and I am also very interested in seeing your use case, which involves InfluxDB and Grafana.

Knowing that Grafana has its default port and 8086, I can see why that is a major issue.

But currently, I see no reason why the XDK should have problems connecting to port 8086 specifically. This makes it all the more strange seeing that an older version of the SDK does this correctly.

Could you tell me, what kind of connection you are attempting to establish? Are you using the low level simplelink API? The error -111 would indicates the error Connection refused on socket level.

If you are using any other API, could you tell me which you are using? Additionally, have you tried to achieve a connection to the same port on another server? Even a Raspberry Pi would do, just to see if this is perhaps a problem with how the server reacts to the XDK.

I have tested both, a UDP connection (using simplelink API) and a TCP connection (using HTTP, which is based on TCP, for convenience) to port 8086 from the XDK to a Raspberry Pi. In both cases, connection and data transfer from XDK to server are working.

Unless I find a way to reproduce this, based on further information from you, it would be difficult to find a solution for this, other than using another port on WB 3.x.

Kind regards,
Franjo

0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 17 오후 6:05 as a reply to Franjo Stjepandic.

Thanx for the feedback. 
I think i have to write the code new with the new WB3.x and RestClient.
Strange,  it compiles error free on WB1.7 and WB3.x and starts perfect.
But only on WB 1.7 Ports other than 80 are working.

So far so good - at the moment we are happy that all runs fine.

First snapshots are now online.

Also - i have seen a great solution from Mr. Christian Renz,
he has a wonderfull Grafana XDK Dashboard.
Another solution i can make is grabing the XDK sensor data from my MQTT broker and "telegraf" them to the influxdb and grafana.
So there are many ways to go...

Greetings Achim

+1 (1 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 18 오전 9:29 as a reply to Achim Kern.

Hello Achim,

since you are using the RestClient from the ServalStack API, the issue may lie in one of the additions from 2.0.1 to 3.0.0.

The variable RestClient_ReqInfo_T has been extended by three fields. They are host, rangeLength and rangeOffset. They are used to define the Host header and the Range header.

The issue is, since the message struct is not initialized to zero values, the host and the rangeLength fields may have arbitrary values, and the message will actually include them.

This can be avoided by either using some meaningful values for these fields or by disabling them.

They can be disabled by setting host and rangeLength to NULL as follows:

RestClient_ReqInfo_T request;

request.rangeLength = 0;
request.host = NULL;
Alternatively, you could do the following, to set the entire struct to zeroes before filling it with values:
RestClient_ReqInfo_T request;

memset(request, NULL, sizeof(RestClient_ReqInfo_T));

// fill request now
Could you try this out with port 8086 and see if that makes it work? As far as I remember, if host is set to NULL , it will be set to the actual destination address.

While there are other solutions one can use as a workaround to this issue, such as using MQTT as an intermediate protocol, it would be useful to know if there is a more straightforward solution to this issue. This would be helpful in case there are other users experiencing the same issue.

Kind regards,
Franjo

 

0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 18 오후 6:17 as a reply to Franjo Stjepandic.
Hello - here is my actually restclient code.
What have i to do now - where should your code snipet fit in ?
This codes runs perfect under version 1.7 - see new snapshot from grafana.
 
 
/* header definition */
#ifndef _XDK_RESTCLIENT_H
#define _XDK_RESTCLIENT_H
/* interface header files */
#include <Serval_Types.h>
/* public interface declaration */
/* public type and macro definitions */
/* public global variable declarations */
/* public function prototype declarations */
retcode_t xdkRestClientInit(void);
void xdkRestClientPostMessage(const char* uripath, char* payload, uint32_t size);
void xdkRestClientDeinit(void);
/* inline function definitions */
#endif /* _XDK_RESTCLIENT_H */
 
 
/* system header files */
#include <stdint.h>
/* interface header files */
#include "simplelink.h"
#include "PAL_initialize_ih.h"
#include "PAL_socketMonitor_ih.h"
/* own header files */
#include "xdkRestClient.h"
#include "xdkConfig.h"
/* constant definitions */
#define MSG_MAX_SIZE  (2048)
/* local variables */
Ip_Address_T destAddr = UINT32_C(0);
char msg_buf[MSG_MAX_SIZE];
/* global functions */

/**************************************************************************
* Function:   retcode_t xdkRestClientInit(void)                     *
* (global)          intialize PAL and network                             *
***************************************************************************
* Created by:  Achim Kern <achim.kern@keho-software.com>             *
* Copyright:  (C) 2017 KeHo Software                                *
* License by:  Bosch Connected Devices and Solutions GmbH, 2017      *
* Update:   03-01-2017 - first code developed                     *
*                              - clean up some entries                    *
* ************************************************************************/
retcode_t xdkRestClientInit(void)
{
  retcode_t rc = RC_OK;
  rc = PAL_initialize();
  if (RC_OK != rc)
  {
    printf("[x] Bosch XDK - PAL and network initialize %i \r\n", rc);
 return rc;
  }
  PAL_socketMonitorInit();
  if (RC_OK != PAL_getIpaddress((uint8_t *) DEST_SERVER, &destAddr))
  {
    printf("[ ] Bosch XDK - failed to get destination IP address\r\n");
 return rc;
  }
  printf("[x] Bosch XDK - initialized RestClient \r\n");
  return RC_OK;
}
/**************************************************************************
* Function:   void xdkRestClientPostMessage(const char* ...)        *
* (global)          post the sensor message payload                       *
***************************************************************************
* Created by:  Achim Kern <achim.kern@keho-software.com>             *
* Copyright:  (C) 2017 KeHo Software                                *
* License by:  Bosch Connected Devices and Solutions GmbH, 2017      *
* Update:   03-01-2017 - first code developed                     *
*                              - clean up some entries                    *
* ************************************************************************/
void xdkRestClientPostMessage(const char* uripath, char* payload, uint32_t size)
{
  int16_t SockID = 0;
  SockID = sl_Socket(SL_AF_INET, SL_SOCK_STREAM, 0);
  if (SockID < 0)
  {
    printf("[ ] Bosch XDK - fail to create socket \r\n");
  }

  SlSockAddrIn_t Addr;
  Addr.sin_family = SL_AF_INET;
  Addr.sin_port = sl_Htons((uint16_t) DEST_PORT_NUMBER);
  Addr.sin_addr.s_addr = destAddr;
  int16_t rc = sl_Connect(SockID, (SlSockAddr_t *) &Addr, sizeof(SlSockAddrIn_t));
  if (rc < 0)
  {
    sl_Close(SockID);
 printf("[ ] Bosch XDK - fail to connect (%d) \r\n", rc);
 return;
  }

  int16_t msg_size = snprintf((char*) msg_buf, MSG_MAX_SIZE,
   "POST %s HTTP/1.1\r\n"
   "Content-Type: application/json\r\n"
   "Connection: close\r\n"
   "Host: %s:%d\r\n"
   "Content-Length: %ld\r\n"
   "\r\n"
   "%s"
   "\r\n", uripath, DEST_SERVER, DEST_PORT_NUMBER, size, payload);
  int16_t numSent = sl_Send(SockID, msg_buf, msg_size, 0);
  if (numSent < 0)
  {
 sl_Close(SockID);
 printf("[ ] Bosch XDK - fail to send message\r\n");
 return;
   }
   printf("[x] Bosch XDK - sent a message (size = %d) -> %s \r\n", msg_size, msg_buf);
   sl_Close(SockID);
   return;
}
/**************************************************************************
* Function:   void xdkRestClientDeinit(void)                        *
* (global)          post the sensor message payload                       *
***************************************************************************
* Created by:  Achim Kern <achim.kern@keho-software.com>             *
* Copyright:  (C) 2017 KeHo Software                                *
* License by:  Bosch Connected Devices and Solutions GmbH, 2017      *
* Update:   03-01-2017 - first code developed                     *
*                              - clean up some entries                    *
* ************************************************************************/
void xdkRestClientDeinit(void)
{
  ;/* DO NOTHING */
}
0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 19 오후 3:23 as a reply to Achim Kern.

Hello Achim,

I actually misunderstood. When you mentioned RestClient, I assumed that you meant the RestClient API provided in the ServalStack library at SDK > xdk110 > Libraries > ServalStack.

Since you actually wrote a connection and are sending a REST message on simplelink (i.e. socket level) yourself, my previous advice does not work of course, since my advice was based on the assumption that you used the ServalStack.

And, since the simplelink implementation files have not changed from XDK-Workbench version 1.7 to 3.1 at all, every function that is prefixed with sl_ should still behave as it behaved before. The only possible point of error regarding this application may actually be the calls of PAL_initialize(), PAL_socketMonitorInit() and PAL_getIpaddress.

There is the possibility of using simplelink API to retrieve the IP address from a hostname instead of PAL_getIpaddress. I could provide you with a code example, if you would like to try that out. That code would essentially use the function sl_NetAppDnsGetHostByName() provided by the interface netapp.h, which is also part of simplelink.

Other than that, I really do not see what could be the reason for the connection's result to be different from 1.7 to 3.1, I am sorry. Perhaps there is something else in the rest of the code that I cannot see, but I do not think that is the case.

Please tell me if this was helpful and if whether you would like to try using the function I mentioned. Otherwise, feel free to ask if you have further questions.

Kind regards,
Franjo

0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 19 오후 6:02 as a reply to Franjo Stjepandic.

Thanx for the feedback.
Would be great if you could provide that code snipet.

The port problem was one point, we also have had to solve
LED, Button, Accoustic Sensor, external connections etc.
all that stuff works fine under 1.7
- now this code is history.

So - finally i think i am happy to have a perfect running system under 1.7.
We try to fix the port problem hopefully next week with your solution idea.

If i have a little bit more time -
i will write the code completely new under version 3.2.

We have provided the influxdb and grafana info today
in the ALEXA news flashbriefing skill
and also under our website

http://www.keho-software.com/xdk_blog.html
 

Greetings Achim



 

0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 22 오후 12:59 as a reply to Achim Kern.

Hello Achim,

the following code snippet will retrieve the IP of a host using simplelink API.
 

_u32 ip;
_i8 * address = (_i8 * ) ADDRESS;
sl_NetAppDnsGetHostByName(address, strlen(ADDRESS), &ipAddr, SL_AF_INET);
In this code snippet, the destination address will be retrieved from ADDRESS and stored as an unsigned integer in ip, usable by other simplelink API. The ip value can be directly stored in Addr.sin_addr.s_addr . ADDRESS is a simple macro representing a hostname-string, such as
 
#define ADDRESS "www.google.com"
// also accepts ip-address as string, such as "192.168.1.1", resulting in 0xc0a80101
// make sure the address is not prefixed with "http://" or anything similar.
The entire code snippet removes the need for using PAL_initialize() , PAL_socketMonitorInit() and PAL_getIpaddress() .

I hope this will solve the issue with the port, since my most likely guess is that the function call PAL_getIpaddress() is not retrieving the IP as expected. Otherwise I would actually recommend to take a look using Wireshark , to see if the message is sent as expected (with the correct payload, headers, destination address and port).

Kind regards,
Franjo

 

0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 28 오전 10:08 as a reply to Franjo Stjepandic.

The hint with the http:// was the solution
now i only use the IP Addr and that works now fine.
This is a test under 3.0.1 WB - try now the other ones next week.
 

Booting application...
 INFO | XDK DEVICE 5:  Jumping to application
 INFO | Port 'COM6' has been disconnected
 INFO | Connecting to XDK device 'XDK Device 5' in port 'COM6'...
 INFO | Connection to port 'COM6' established
 INFO | XDK DEVICE 5: [x] Bosch XDK - setting IP to DHCP
 INFO | XDK DEVICE 5: [x] Bosch XDK - connecting to KeHoSoftware
 INFO | XDK DEVICE 5: [x] Bosch XDK - getting IP settings
 INFO | XDK DEVICE 5: [x] Bosch XDK - convert the IP address to string format
 INFO | XDK DEVICE 5: [x] Bosch XDK - connected to WPA network successfully.
 INFO | XDK DEVICE 5: [x] Bosch XDK - IP address of your XDK device:7CEC7902E5A5 - 192.168.178.101
 INFO | XDK DEVICE 5:  IP address of 192.168.178.99 is 192.168.178.99
 INFO | XDK DEVICE 5:  [x] Bosch XDK - initialized RestClient
 INFO | XDK DEVICE 5: [x] Bosch XDK - intialization complete!
 INFO | XDK DEVICE 5: [x] Bosch XDK - sensor payload string = humidity,device=xdk value=31
 INFO | XDK DEVICE 5: [x] Bosch XDK - sent a message (size = 161) -> POST /write?db=mydb HTTP/1.1
 INFO | XDK DEVICE 5: [x] Bosch XDK - sensor payload string = humidity,device=xdk value=31
 INFO | XDK DEVICE 5: [x] Bosch XDK - sent a message (size = 161) -> POST /write?db=mydb HTTP/1.1
 INFO | XDK DEVICE 5: [x] Bosch XDK - sensor payload string = humidity,device=xdk value=31
 INFO | XDK DEVICE 5: [x] Bosch XDK - sent a message (size = 161) -> POST /write?db=mydb HTTP/1.1

0 (0 투표)
RE: XDK InfluxDB and Grafana
응답
18. 1. 29 오후 1:05 as a reply to Achim Kern.

Hello Achim,

glad to hear that you were able to solve the issue.

the prefix http:// is an extra information for some programs, such as an internet browser, to let them know which port to use, since most protocols have a standard port. For example, http has the standard port of 80, while ftp has the standard ports 20 and 21. A browser will use ports automatically, when a prefix is added.

But for a DNS server, this information is not neccessary, since the address is the payload, and the port is and entirely different one. And obviously, the DNS functions of PAL and simplelink do not expect the prefix, and can thus not handle it.

Anyhow, if you have any further questions, please feel free to ask.

Kind regards,
Franjo

0 (0 투표)