|
I agree that REST seems like a poor choice for low latency.
Just a thought - latency can be reduced by using more modern frameworks that while they do serve the entire state initially only subsequently serve the elements of the state that have changed or use SignalR.
On the area of latency - even if you do use a protocol that is less data intensive than REST, if you are not using a private direct network connection your latency will surely always be at the mercy of things that are out of your control - or have I misunderstood what you are saying?
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
|
|
|
|
|
It's not up to me, we are at the mercy of whatever the device manufacturer decides to do. Of course TCP/IP in general is a bad choice for most small automation doo-dads, but since it's the defacto, ubiquitous connection scheme that is going to be in there in the home, more and more companies use it. It's always the same, unplanned, excremental growth leads to the worst case scenario becoming the standard scenario so often.
The only device that uses HTTP well (that I've run across) is the ISY 994. It uses a persistent HTTP connection and just asynchronously sends notifications as GETs to the automation system when something changes. If you are going to use HTTP, that's how it should be done.
There are other simple schemes that work well. One is to keep a running 64 bit serial number. Every time a value is changed, bump the serial number and set it on that value. Keep a sorted list of such changes (Which is simple and efficient to do, push each one onto the top of a stack when it changes, if it's full dump the bottom one.)
When a client asks for changes, they pass the last serial number they got (zero if none yet), and the device can just find that number in the list (binary search since it is sorted by serial number), and returns him anything from there forward. If the number is below the bottom, he returns an error saying 'sorry you have to do a full resync'. If it works, the automation system just stores that returned serial number as the last one it got, and passes it back next time. The majority of queries just return a very small 'no changes' status since the serial number passed is the same as the one on top of the stack.
I use this type of scheme to VERY good effect in CQC in a number of places. My ORB supports it as well, with a special 'poll method' that takes a serial number as the first parameter and returns a boolean. If the serial number is the same as the handler on the server side, then it returns false and the server side ORB knows it doesn't have to stream back any of the output parameters.
Explorans limites defectum
|
|
|
|
|
|
Dean Roddey wrote: Of course TCP/IP in general is a bad choice for most small automation doo-dads
This is out of my level of experience however I am aware that UDP is an alternative in this sort of thing.
However the issue with UDP is that there is no concept of packets being dropped or guarantee that what gets sent from the server will get to the client in once piece - see what I mean about 'out of my level of experience", I almost started referring to the interwebs as a series of tubes.
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
|
|
|
|
|
Actually I meant IP in general. Most home automation stuff should be on a dedicated backbone, like Zigbee (or in the case of some of the higher end stuff like Lutron, they are on proprietary wired or wireless protocols.) These are purely local systems that cannot be attacked by every scumbag on the network.
Some stuff needs higher bandwidth than that, and they are OK using TCP/IP if well done and most of the pro level stuff is. A lot of consumer stuff very much isn't. For instance cameras generally need to be on the local network for video streaming purposes, but they are horrible security-wise in general. There's been so many security issues reported with them. And all these small companies doing little IoTs doodads probably don't remotely deal with security in any comprehensive way. They are probably struggling to get the product done before they run out of money.
You really need to cordon off that stuff on a separate LAN to be safe, but then of course they can still be used in DOS attacks unless you prevent them from making outgoing connections. And your average user isn't competent to do either of those things.
Explorans limites defectum
|
|
|
|
|
I think you need to give it a rest...
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
One of the better rants I've come across.
I think in general you are ranting about :
Design Versus Defaults!
In other words, why do people supposedly design devices that are not actually designed at all but are just knee-jerk reactions to a problem.
Someone recently told me they he "Finally got my garage door set up on wifi. It took me a long time though.".
I shivered a bit as I imagined his garage door being opened by random hackers on the Internet.
It makes no sense to create a garage door opener that uses wifi and connects to the Internet.
He also mentioned that it takes a bit for his phone to connect to his wifi anyways so it takes a while to use it each time.
But many manufacturers just default to wifi when they should just use local wireless like bluetooth.
I created such a device and it was really quite easy and very inexpensive (Never Buy A Garage Door Remote Again: Open Your Door With Your Android Phone (via Bluetooth)[^]).
Also, even if you want to know if your garage door is up or down from a remote location that is still possible without forcing the user to connect his garage door to the Internet.
Well, that is design by default instead of design through thinking and choosing the proper devices and services.
|
|
|
|
|
Oh, the whole home automation thing, as it has become in recent years, is just a security nightmare, no doubt. All these small doo-dads from China that people just plop inside their local network and they are connecting to who knows what, or being infected and used in massive DOS attacks and such.
The thing is, a lot of these newbies will shat on serial connections as old fashioned, but they are inherently safe, point to point connections between the automation system and the device. As long as the bandwidth is sufficient and the devices relatively close (and often they are) then it's an excellent connection type to use (and very cheap for the manufacturer.)
But of course most of these newer companies are not interested in that, they are interested in tying customers to their cloud-based services so that they can collect data and monthly fees. And many of them don't look beyond their own phone based app control interfaces, so we end up going back to the start except that now there are a bunch of virtual remote controls on a phone instead of a bunch of real remote controls on a coffee table.
Explorans limites defectum
|
|
|
|
|
raddevus wrote: One of the better rants I've come across.
PFFFFT!!! He ain't serious unless he's said "f*ck" at least once.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
Gosh darn the blankety heck.
Explorans limites defectum
|
|
|
|
|
|
|
We go through that sort of thing occasionally, as we build commercial ink-jet printing systems. I can't imagine anyone using some godforsaken web protocol for command/status in any kind of process control environment. All of our peripheral systems use TCP/IP socket communications, usually over a gigabit private Ethernet.
Given that we're running paper at up to 17 feet per second, half-second latency turns into a lot of paper.
Software Zen: delete this;
|
|
|
|
|
In the so-called 'internet of things' world, which is basically what has become of automation in the popular press and hence in the great unwashed masses for the most part, it's way too common. And of course it's even worse in that half the time it's a cloud based API, so not only REST but you have to talk to a server half-way around the world to talk to a device 20 feet from the computer.
Explorans limites defectum
|
|
|
|
|
Dean Roddey wrote: you have to talk to a server half-way around the world to talk to a device 20 feet from the computer
Same deal where I work. They removed (or refused to replace) all of our attached desktop printers. We now have department printers, all managed from the home office. When I print a code fragment to mark up by hand(*), the data goes 800 miles to a server and then comes back 800 miles to the printer 30 feet away from me. This process typically takes 3-5 minutes.
(*) Yes, I'm an old fart. I still do this occasionally. Don't worry, though - my annual stack of recycled paper is around one inch tall.
Software Zen: delete this;
|
|
|
|
|
I find REST to be carp... but i'm in the minority... so i have to use it 
I once wrote something, but did not focus on latency. Its a good point that i had not considered.
GitHub - maxsnts/TARAPI-An-API-Rant: “TARAPI” An API Rant![^]
(I end up not finish the Rant because by that point i had released some steam.
There's even lots of spelling mistakes)
|
|
|
|
|
<sarcasm>
Well duh, you need to use npm to install AngularJS so you can use Node.js for your WebSockets that you should connect using Webpack.
If you're not using at least 100 libraries you're not putting in enough effort and YOU're the problem 
</sarcasm>
|
|
|
|
|
Does one of those libraries use the Retroflexive, Inter-secular Fractal Pipe Massage pattern?
Explorans limites defectum
|
|
|
|
|
Is your problem with REST per se, or is it just a problem with slow devices transferring too much state?
|
|
|
|
|
The problem is as I originally outlined. It's a completely synchronous call/response system being used in a situation where 99.9999% of the time there's nothing to respond with. But, when there is, it needs to be seen very quickly. So it's just the completely wrong tool for the job.
On the automation system side, you could be dealing with hundreds of devices, so it's doing huge amounts of I/O and network traffic for essentially no reason, and it has to parse all that data, convert to binary, and check it against what it saw last time to see if it has changed, and each of those devices might have quite a few values to deal with. If the device can only transfer a single big chunk of state, it just makes it that much worse.
If multiple clients needs to do the same on those devices, it compounds, and sometimes overwhelms the device.
Explorans limites defectum
|
|
|
|
|
Sorry, I had my HTML backward. Your computer should be the server and all the IoT thingamabobs should be clients. Then they could PUT or POST asynchronously. If that's not the way the world works, get another world.
|
|
|
|
|
That's generally not desirable for security reasons. The automation system should be the one making connections, as configured by the administrator. In some cases it's unavailable but typically it's something that's not done unless really necessary.
Explorans limites defectum
|
|
|
|
|
Dean Roddey wrote: Why don't people get that REST sucks for a wide variety of applications
Or from my point of view why don't people get that Agile doesn't work for architecture?
Dean Roddey wrote: People make a device, and probably make no effort to ask anyone like me what are needs are,
They decide that they can deliver the API in 6 weeks or worse 2 weeks, and the "product owner" is more interested in delivery than usability. Not to mention of course that when the implementation phase starts there is in fact no actual requirements beyond "make an API" but certainly no willingness to back off on the delivery.
Dean Roddey wrote: So now you have a a bunch of devices out that that you just have to pound relentlessly with GET commands ...make it 'easy' by only allowing the transfer of the entire state of the device
It was designed based on what info the implementer had. They didn't have time to make 120 different routines, and QA certainly would not have had time to test it. And probably when asked they asserted they were not even sure anyone would use the API. So you get one API call.
Dean Roddey wrote: It's like so many people just don't even think beyond a REST API or HTTP, and throw it into everything, whether it's remotely appropriate or not.
Its like people think that 2 week sprints are more than sufficient to plan for an architecture that is supposed to be robust and last for years. An an off hand comment in a meeting several weeks ago leads to a requirement that must be implemented right now. And it certainly doesn't have anything to do with actually determining how it will be used and very definitely there is no time to actually figure that out.
|
|
|
|
|
Do cows have hooves because they lactose?
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Are all bulls horny?
"If we don't change direction, we'll end up where we're going"
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
