HTTP Client Unreliable
응답
17. 8. 1 오전 2:45

I am having trouble getting http requests to be sent reliably. I am using the httpExampleClient code and have only changed the destination ipAddress to direct them to a server on my computer. I can send requests without an issue from other devices but the XDK is very unreliable (Only about 20% of requests are successful). Connecting to the wifi network appears to be working fine. I am getting various errors sometimes saying errors occurred in downloading html, others saying error ocurred in connecting to the server. I am using workbench 2.0.1

 

Any ideas what could be causing these issues?

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 1 오전 11:53 as a reply to Aidan McCoy.

Hello Aidan,

Welcome to the XDK community.
Your issue with the HttpExampleClient can have multiple causes. To solve them you have to distinguish the errors you are receiving in the XDK-Workbench console. The issue error occurred in downloading HTML only tells you that the XDK was unable to receive the requested data. In most cases, the issue is caused by a wrong configured content type in the HttpExampleClient example. To solve it, you only need to change the content type in line 159 in the implementation file HttpExampleClient.c.

Regarding your other issue error occurred in connecting server, could you please tell me if the XDK is still connected to your local Wi-Fi when this error appears? The most common cause for the XDK to be unable to send further HTTP requests, is the XDK losing the Wi-Fi connection. The HttpExampleClient example has no integrated implementation to reconnect to the Wi-Fi.

Regarding your of the reliability of the HTTP requests, I tested the HttpExampleClient example with a local HTTP server over an hour and I am still receiving GET requests.

Because of this, I assume that it really is a loss of connection, probably because of weak signal strength.

In any case, I would ask you to verify if the XDK has lost the Wi-Fi connection as soon as you're not receiving error messages anymore. You can do this with the terminal command "ping your-xdk-ip" from any other device within the same WiFi.

Please let me know if this was helpful and do not hesitate to ask if you have further questions.

Kind regards,
Franjo

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 1 오후 9:35 as a reply to Franjo Stjepandic.

Thank you for the response, regarding the Error occurred in downloading html, what should I change the content type to? I have it as html currently but also changed it to Http_ContentType_Text_Plain as I am just returning a short string. I recieve the string correctly on a successful GET request and it is printed to the console with a correct length as well.

Regarding connecting to the server, the XDK is not dropping wifi. It never stopps sending all messages, just only most of them never reach the server. I am able to successfully ping it at any time with 0% packet loss. My XDK is sitting a few feet from my wifi router so I do not think that weak signal would be an issue as none of my other devices are seeing this problem.

I am also seeing Failed HTTPCLient_initRequest pretty regularly as well if this helps as well.

Thanks,

Aidan

EDIT: Actually I was wrong, I am having trouble pinging the XDK, but the signal strength should still be strong. Any other reason it would be losing connection?

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 2 오후 2:22 as a reply to Aidan McCoy.

Hello Aidan,

I am glad to hear that you were able to solve the issue regarding the content-type. As for the loss of WiFi-Connection of your XDK, the reason could be that the hotspot is temporarily overburdened. This happens when too many clients are connected to the hotspot. In this case, the XDK might disconnect from it.

To confirm it, I would ask you to use another hotspot with only a small amount of connected clients and then to test the reliability of your HTTP requests again.

In any case, implementing an auto-reconnect functionality would probably solve this issue.

Kind regards,
Franjo

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 2 오후 7:24 as a reply to Franjo Stjepandic.

So I am actually seeing the same issues when I am using our work network (Lots of devices), my home network (A few devices) or hosting a separate network on my computer with only the XDK connected to it. 

That being said, I would also like to create an auto reconnect function, what would be the best way to do this? I am assuming just setting a flag whenever more than one or two consecutive errors occur and call wlanConnect again?

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 3 오후 1:16 as a reply to Aidan McCoy.

Hello Aidan,

that is basically the core idea of reconnecting. Another option would be to check the current network status in every loop. The code-snippet for this would be:

if(CONNECTED_AND_IPV4_ACQUIRED != WlanConnect_GetCurrentNwStatus()){
    WlanConnect_SSID_T connectSSID = (WlanConnect_SSID_T) WLAN_CONNECT_WPA_SSID;
    WlanConnect_PassPhrase_T connectPassPhrase = (WlanConnect_PassPhrase_T) WLAN_CONNECT_WPA_PASS;
    NetworkConfig_SetIpDhcp(NULL);
    if (RETCODE_OK == WlanConnect_WPA(connectSSID, connectPassPhrase, NULL)){
        // RECONNECTED
    }
}

This will basically attempt to reconnect every time the network status is not "connected and ipv4 acquired".

Please tell me if this was helpful, and do not hesitate to ask further questions.

Kind regards,
Franjo

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 3 오후 9:20 as a reply to Franjo Stjepandic.

I added this to my code with a print statement when it successfully reconnected and it never seems to be called. I think it is staying connected to wifi. What can cause the httpClient_initRequest to fail?

EDIT: When calling the httpClient_initRequest, I am recieving error code 1100 for RC_TCP_NOT_CONNECTED. This occurs even after successfully sending a message. I understand the basics of TCP connections. Does it create a new session every time it wants to send a message or should I be using the same session each time?

EDIT2: Also, is there a timeout between messages waiting for a response that would block other messages from sending? Something that if one message failed it would fail the next couple attempts til it times out and creates a new session

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 4 오후 1:20 as a reply to Aidan McCoy.

Hello Aidan,

yes, the HttpExampleClient creates a new session everytime you send a message, and this is perfectly fine. HTTP is based on the REST paradigm, and this roughly implies that TCP sessions are needed for only one message, since every message "is" a session.

Concerning your other question, the XDK can handle multiple TCP sessions at once. HTTP messages should not be able to block each other out, or at least there would be a completely different return error code.

Anyhow, at this point the XDK can be ruled out as the source of this issue. It doesn't disconnect, and it keeps on trying to send requests. The fact that it receives the error code for RC_TCP_NOT_CONNECTED indicates that it cannot connect to the server.

I tried the HttpExampleClient with three different servers: (1) A simple python HTTP Server on my Ubuntu PC (simply echoes the header and responds with code 200), and the addresses (2) httpbin.org/ip and (3) xdk.bosch-connectivity.com/.

For (1) and (3), every request was successful and I got a response for every request. For (2), many of the requests failed. This supports my assumption, that it might be server dependent.

Have you tried sending GET requests to any other server (for example xdk.bosch-connectivity.com/), to see if you get the same behaviour? What kind of server are you running on your PC?

Kind regards,
Franjo

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 7 오후 10:45 as a reply to Franjo Stjepandic.

Alright that makes sense.

After further testing, I was able to get my get requests out to an external server, both posttestserver.com and xdk.bosch-connectivity.com. How were you able to get httpbin.org/ip to work? When translating the url to an ip address, it was returning different ip addresses and none of them were correct. 

The server that I am working with just returns a short string corresponding to the uri sent to it. I thought the issue was on the XDK side because I can spam the server with requests from my postman or from my phone and I have no issues with getting correct responses. Each time the request from the XDK reaches my computer, I get a correct response from the server. Just when I get the Failes HttpClient_initRequest error is when the message is not sent through.  What exactly is happening in the httpClient_initRequest call?

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 8 오전 10:39 as a reply to Aidan McCoy.

Hello Aidan,

since you were able to send requests to posttestserver.com and xdk.bosch-connectivity.com, did you get the same behaviour as with your own server, with HTTP requests failing most of the time? At least for xdk.bosch-connectivity.com, that shouldn't be the case.

For httpbin.org, I simply used the address as is. In the HttpExampleClient's source file HttpExampleClient.c, in the function init, the function PAL_getIpaddress() is called, with the host-name as the input (the host-name can be  httpbin.org, for example). This function retrieves the IP, so you don't have to do it yourself. You can also insert an IP-string instead of the host-name, which will work just the same. Additionally, to address the URL /ip, there is a variable called url_ptr inside the function connectServer(). Change this to /ip to send a request to httpbin.org/ip.

Regarding the function httpClient_initRequest(), it has the following outline:

First, it tries to allocate the Msg struct. If there is not enough memory available, RC_HTTP_CLIENT_INIT_REQ_FAILED is returned.

Second, it checks whether a session with the destination server is already in progress, RC_HTTP_CLIENT_PENDING_CONNECTION is returned and the message aborted.

After the message and the session are allocated, the function tries to connect to the destination server via TCP, using the function Tcp_connect(). Now I'd have assumed that this is probably where you get the error message. But, if Tcp_connect() fails, the return-code should be RC_HTTP_CLIENT_INIT_REQ_FAILED. I cannot exactly tell, where the RC_TCP_NOT_CONNECTED return-code is used, but I make the following conclusion:

Before error occurred in connecting server is printed, the XDK can actually initialize the request and push it successfully, but the request eventually times out. And before the session expires, the application tries to initialize another request, which results in RC_TCP_NOT_CONNECTED. I said earlier that the HTTP Client can handle multiple sessions, but this is only true if the destinations are different. It can only handle one session per destination. Therefore, what you said is true, the failed requests do actually block out the following request(s).

To go into more detail, when pushRequest() is called, the request is attempted to be sent, and the function callbackOnSent() is called when the sending-process is done. The process is done, when the message either got sent successfully, or when it timed out. The callbackOnSent() also receives a return code, and it is 1357, which translates to RC_HTTP_SEND_ERROR, when the message fails.

Basically, the message is attempted to be sent, which times out after roughly 10 seconds. Then, roughly 7 seconds later, the session is finally closed.

In any case, the HTTP API is actually behaving as intended. The problem is simply the TCP connection breaking somehow, during or before the sending-process, and as I mentioned before, I assume that this is somehow a problem with the server, since some servers seem to have these problems, while others do not. This is difficult to diagnose, and I can only make guesses as to why this occurs.

Kind regards,
Franjo

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 10 오후 7:28 as a reply to Franjo Stjepandic.

Thank you so much for the lengthy response, this is very helpful figuring out how this works. 

As far as sending requests to httpbin.org/ip I had been doing what you suggested but it still wasn't working. I was getting directed to a site hosted by heroku saying that the site did not currently exist. Not sure what the issue is here but it is not too important in the longrun. 

The way you explained the httpClient_initRequest() makes sense why it would be failing there then. I will try to work on the server side to figure out what is causing these errors. That being said, the response back to the XDK is not overly important as it is currently only used for sending sensor data to the server. Is there a way to close the TCP connection from the XDK side that might allow it to open a new session before the previous session times out?

Thanks again for the help.

Aidan

0 (0 투표)
RE: HTTP Client Unreliable
응답
17. 8. 11 오후 5:02 as a reply to Aidan McCoy.

Hello Aidan,

I am glad that I could help you.

Further I took a look at the implementation of the HTTP API, to see if it would be possible to force a rapid closing of the TCP connection.

Unfortunately, it is not possible to do that manually, because the necessarily needed information parameters about the TCP connection are generated internally during calling of Http_initRequest() and cannot be accessed from outside.

I guess the only workaround to solve this, as you already mentioned, is to figure out on the server side, what causes the loss of the TCP connection.

Kind regards,
Franjo

0 (0 투표)