Read NMEA from Server

[agraham]

I am at a loss! I can confirm that when connected to your server when DataAvailable is true it can take a long time for ReadStream to return when polling quickly, but it just shouldn't do this, it should return immediately with whatever data is available.

I have knocked up a server to send a long string every 2 seconds and connected my net demo app as a client from another computer to it over my network. The timer tick is 100mS in the client and it behaved exactly as I would have expected, displaying the new string every 2 seconds. Connect the same unchanged client to your server and it locks up. There really should be no difference as far as the client is concerned, it is just waiting for a long string to arrive on a network port. The only thing I can think of is that there is something weird with the TCP/IP stack in that server.

Sory but I can't think of anything else to do!

[schimanski]

Thanks for your efforts, agraham....

[schimanski]

Hello!

Thanks, agraham, for your last threads. I think, I have to take another way.

So, i have a new idea. Is it possible to run a seperat small application in the backround that takes the data from the server and send this data to the main application? So the small application can stop, when it is waiting for data without lock up the main program.

[agraham]

I would try using my Threading library Threading library for optimising compiler to run what is presently your timer code on a thread passing the data to the GPS. The GPSDecoded event is thread safe and will tell the rest of your code that data has arrived. Your thread can use Sleep() to determine the approximate rate you want to check DataAvailable. You should use RunLock in both main program and thread to access the GPS object.

I can't help but think there is something wrong somewhere - you shouldn't need to go to these lengths to get it to work

[TWELVE]

Hello schimanski,

i just flew over your code.

From what i can understand so far, your problem is that you queue up data from the server if you don't read the net buffer for a while.

I don't know what kind of server this is and what protocol it's using, but my first guess is, to just close the connection if you don't want to read data anymore.

If you request data from the server and the server is sending reply data to your request continously, it is obvious that you need to read all reponses before you can get to the newer data.Furthermore, new data will be on the end of your buffer and therefore if you just read the first bytes / characters of your buffer you will always get the last server reponse the client received when it stopped reading from the stream.


Receivebuffer:

NMEA sentence 1, NMEA sentence 2

You read the sentences and therefore the buffer is now empty.
Server sends new data and you read the receivebuffer constantly.The data in the first characters / bytes is always current due to this.

You stop then reading the stream.After 2 minutes you start to read the stream again.

The receivebuffer does now look like this:

NMEA Sentence 1 ( 2 minutes old !) NMEA Sentence 2 ( 2 mins - 1sec) NMEA Sentence 3 ( 2mins - 2 secs.)

If you now just look at the first characters / bytes of your buffer, you will see only the old data, to get newer data sentences, you have to look at the end of your buffer, for example by searching strings or characters.






cheers

TWELVE

[schimanski]

Hello TWELVE!

Thanks for your fast answer, but that was not the problem, which I mean. Sorry, there are al lot of problems in my thread.

I mean this:

PHP Code:


I have written a smal testapp to see, where the problem is. 
When the server sends no data, the application will stop as long,
as the server sends data again. When there is data again, the Application 
runs without any problems. Is there no data, i only can stop the app with the
task-manager or break down the GPRS-connection. Here the code:

Sub App_Start
  Form1.Show
  client.New1
  Client.Connect ("87.139.??.???",11112)
  filestream.New1(client.GetStream,false)
  timer3.Enabled=true 'timer3=500 or 1000 ms'
End Sub

Sub Timer3_Tick
  If client.DataAvailable=true Then
    filestream.ReadString
    label5.Text="data!"
  Else
    label5.Text="no data!"
  End If
End Sub

The server-connection is always available! 


I think, that client.dataavailable on this server (sends NMEA every two seconds as string in a TCP-protocol) also doesn't run. When the server stop sending data, the application lock up in the line filestream.readstring.

Is there a way to write another sub with a counter etc.?

The device is connected with the server over an GPRS-connection.

m.f.G.

schimanski

[TWELVE]

but that was not the problem, which I mean.

Sorry..sometimes hard to get that sorted out in these longer threads.

I think, that client.dataavailable on this server (sends NMEA every two seconds as string in a TCP-protocol) also doesn't run.

If it wouldn't, you never would get any data back from the server.

When the server stop sending data, the application lock up in the line filestream.readstring.

From my point of view this can never happen, because the filestream.ReadString is only called if the client.DataAvailable condition is true.It can happen, if you don't use the client.DataAvailable thing and just try to read the stream.If there's data you will get it, if not, the .readstring will lock until data arrives the net buffer.

Since the timer3 is only called every half or full second, the main app would
not get locked entirely - if there's no deadlock condition within the timer3 code as described by you.If the receiving of data takes a notable amount of time ( and this is the case without doubt when GPRS is used..) , the main app freezes during this time.

Multi-Threading as suggested by agraham would help out here only, if there's no deadlock condition ( ReadString is just waiting and has a chance to get read new data from the stream ever ).

But certainly the exact behavior of this code depends on the latency/speed of your internet connection.If the connections needs 20secs. to retrieve the data, the Main App / GUI gets blocked for this period of time.

So, my question is: does it really deadlock or just block for a couple of seconds...?

If there's a main app and a code part, that is transfering data from/to internet , it is always a good idea to have that run in parallel ( threading, OS callback etc..) because of the freeze effects.Since Basic4PPC is not able to handle threading, this can only be achieved with agraham's threading lib.

Is there a way to write another sub with a counter etc.?

I don't think a counter can help here somehow.The only thing that is able to prevent the .Readstring from blocking is the client.DataAvailable thing.


I don't think you have to struggle here with the issue i described in the telnet client thread, because in that case your "If client.DataAvailable=true Then" would not become true and therefore the code just would not read from the stream.My code works different, because of the used protocol.I send commands to the server and expect a certain reply to this.



regards

TWELVE

[agraham]

Originally Posted by TWELVE

From my point of view this can never happen, because the filestream.ReadString is only called if the client.DataAvailable condition is true.

It does happen on his server, I've tried it - see post #31.

[schimanski]

So, my question is: does it really deadlock or just block for a couple of seconds...?

It is a really deadlock, not only a couple of seconds.


You're so right. I see, that the problem isn't dataavailable.

I don't have any authority to the server-code, so I think, that I have to life with this problem.

So much thanks to you, TWELVE and AGRAHAM, for your efforts.

[agraham]

This DataAvailable has been worrying me. I have googled around and found a very few instances of other people suffering similar reliability problems with DataAvailable so I have thought of a possible workaround. This needs .NET 2.0 and the Door library. It relies on setting the ReadTimeout of the Stream to a small number then calling Read which then throws an exception on timeout or returns the number of bytes read. I have tried the following code in a real application and it works for me. However I have not experienced any DataAvailable problems myself so I do not know whether this technique will sidestep any problems or not. Anyway being able to try a read and not block if it fails can't help but be a good thing.

' globals
Count = 0 ' count of bytes received
Dim WebResp(4096) As Byte ' read buffer


Sub DataAvailable
  ErrorLabel(DAerr)
  count = Stream.ReadBytes(WebResp(),4096)
  Return true
DAerr:
  Return false ' if the Read times out it throws an exception
End Sub


Sub NetGetData(req)
  msg ="An error occurred connecting to " & URL
  ErrorLabel("err2")
  Client.New1
  Client.Connect(URL, Port)
  Stream.New1(Client.GetStream,true)
  StreamObj.Value = Client.GetStream 'StreamObj is a Door library Object
  StreamObj.SetProperty("ReadTimeout",1) ' set the Read stream timeout to 1 mSec
  msg = "GET /" & req & " HTTP/1.1" & Crlf 
  msg = msg & "Host: " & URL & Crlf
  msg = msg & "Proxy-Connection: Keep-Alive" & Crlf
  msg = msg & "Pragma: no-cache" & crlf
  WebReq() = Stream.StringToBytes(msg)	
  Stream.WriteBytes2(WebReq(),0,StrLength(msg))
  Timer = 0
  Do
    Sleep(SleepMs)
    Timer = Timer + SleepMs
    If Timer >= Timeout Then
       Return  "A timeout occurred communicating with " & Device
    End If
    Loop Until DataAvailable
    msg = Stream.BytesToString (WebResp(), 0, Count)
  Client.Close
  Return msg
End Sub