|
|
I think you're overestimating: video data is pretty well compressed, and video camera data compresses very well as most of the frame tends to be identical to the last - in many cases the image a camera sees won't change for days!
Give it a try: see how much data you get from a single camera in a "busy" location and base estimates for all cameras on that.
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
I agree with Griff - video compression is pretty good these days. You need to note that MOST of your video data will not be changing (think camera staring at hallway all night). This is definitely something to test how it scales.
Need some help? Sounds like an interesting project 
Some other thoughts:
- is it a requirement that you store 168 hours? Does it roll over at that point?
What I'm thinking is the way a dash cam works - it basically keeps the last 30
minutes of video, looping along until you hit the save button. Maybe you could keep
the last 24 hours? - Offsite storage might be best, but you're really going to need local storage for the
initial feed, the OC3 line will cost more than the storage
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Duh. Never thunk it. You been doing this awhile ? You obviously have some active brain cells in this domain.
|
|
|
|
|
I did some work in the past with video compression and real-time video mixing for defense applications. Pretty cool things you can do with the right hardware. These days, there is so much COTS technology. And if you cannot find exactly what you want, it would not be hard to find a h/w engineers to whip up a custom board. Just look at what people are doing with all of the Arduino tech.
The general security companies I've seen are sort of stuck behind the times, but they are starting to get new blood and ideas. But their prices are up there.
One thing if you do dabble in this area - security is all important. There are so many systems open to hackers with an IQ of 80 or less. The developers are just criminal in their negligence. So - all comms have to be encrypted.
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
I've done a bit of many things! 
I've not done any real work into security video, but I know some bits about video generally and I'm good at looking at situations and analysing them, is all.
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
Quote: The quantity I'm considering would not work with on-site storage; this quantity would have to be off-site.
Why?
What is the use case for the data? If it is purely for security, and will only be reviewed after an incident, then majority of it will be written, archived for a week, and finally deleted. Why set up a system to move all that data around? Keep it as close to the camera as possible.
I'd look at a node consisting of simple PC, redundant storage, recording from several cameras, connected to dedicated security (preferably wired) network and accessible from central control station. Easier to add more cameras too.
When access to the video is required, simply copy relevant files based on incident time to a central location.
|
|
|
|
|
NeverJustHere wrote: (preferably wired)
How come ?
Just for reference, in this case at least two different humans will be viewing the same camera, and as many as 25 at one time.
|
|
|
|
|
If someone figures out the security system is dependent on WiFi (or buys the details from a disgruntled ex-employee), then they can jam frequencies or perform denial of service attacks to mess with the system. Wired would not have this problem.
|
|
|
|
|
At some point this could almost be easier and cheaper if you just hired some human to watch the camera and take appropriate action if whatever happens happens...
|
|
|
|
|
I don't where your job is or what it is but does your project have any legal requirement for storage? Where I used to work we had to keep video of all cameras for the last 3 weeks at a minimum and system redundancy.
Due to compression you're not going to see 500GB from one camera. Like has already been said, I recommend testing and monitoring using a busy location and using local storage for a couple of weeks to get a good data set and bandwidth pattern over time.
Depending on requirements and the hardware chosen you may even be able get away with adjusting compression ratios, increasing compression for dormant locations or those not as security sensitive and lowering it for higher traffic/security areas. This won't get you a lot more room to play with but if you run up against limitations, it may help you.
Also, having an off-site storage is problematic for that many cameras. Sure you can use an OC3 (probably going to need it!) but it's expensive as hell and it's a single point of failure. If the line gets cut all your cameras go down.
If off-site is a mandatory requirement, you'd be better served having multiple connections from multiple vendors going to off-site storage. Groups of cameras would use each connection. In that case, if one vendor goes down or connection gets cut, you don't lose all of your camera coverage.
I wouldn't go with an all-in-one system that has 500 camera feeds to it. I'd go with a much smaller system that handles, say, about 50 cameras at once and has it's own storage system. Then you have at least 10 of these systems spread throughout your site(s) making your system modular, expandable and fault tolerant. You can then have a "supervisor" system that manages them and this also gives you the opportunity to start with a smaller implementation and implement it in stages. You also get the time to find and fix bugs along the way before you get to your full implementation.
|
|
|
|
|
Dave Kreskowiak wrote: does your project have any legal requirement for storage?
Now there's a question I didn't think to ask.
I never thought the government would be interested in a vid clip of cars in a parking lot.
What is the typical name of the government agency who gets involved in such requirements ?
Dave Kreskowiak wrote: If off-site is a mandatory requirement,
Thanks for the extra brain cells. I think you guys have convinced me to run a test first and see what kind of data numbers I'm likely to generate.
The off-site thing is a result of me thinking: the humans who will be using the video are definitely not going to be the nerd type. My thoughts are/were that normal humans these days want a website with a simple "click here to view yesterday's tape" sort of thing
|
|
|
|
|
C-P-User-3 wrote: What is the typical name of the government agency who gets involved in such requirements ?
Speaking of taking video of cars in a parking lot, we call that a border crossing. The Department of Homeland Security, U.S. Customs and Border Protection.
|
|
|
|
|
Until they finally tore up the toll booths - Highway 400 in Atlanta 
Airport parking too...
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
C-P-User-3 wrote: What is the typical name of the government agency who gets involved in such requirements ?
The Department of Administrative Affairs?
If you have an important point to make, don't try to be subtle or clever. Use a pile driver. Hit the point once. Then come back and hit it again. Then hit it a third time - a tremendous whack.
--Winston Churchill
|
|
|
|
|
I have a very interesting project about the same line...
First of all the storage not that huge as you estimated as there is compression in the middle too!
In the design we thought to be the final we paired every endpoint with a single board computer (like Raspberry Pi), with a smart software of its own...
So the camera actually stores data on the local single board computer and those computers are connecting to a server farm (with fast and expensive but not too large storage) via a interface (API) that enables them to upload content...
Day-by-day you will be a bit offline, but you can design your system with an emergency line, that enables to some of the cameras to go online...
The data from the server farm downloaded to some backup servers (slower and cheaper) to clear the fast storage for daily work...
Do these things:
1. Group you cameras by importance
2. Sample each group to estimate storage need
3. Design your system's parts
Skipper: We'll fix it.
Alex: Fix it? How you gonna fix this?
Skipper: Grit, spit and a whole lotta duct tape.
|
|
|
|
|
I've briefly worked on such a system. Some include motion sensors along with the camera, while others analyze the video data. As long as no motion is detected, they omit the data from long-term storage, or only store a brief sample (one frame per second/minute, etc.). That, along with conventional video compression, reduces the storage requirements from outlandish to merely huge.
Software Zen: delete this;
|
|
|
|
|
You can also configure most cameras to use 'Motion-Detection' setting. In my experience, most cameras are installed in places where pictures very rarely change.
Behzad
|
|
|
|
|
The system that I've had experience allows you to set how often it captures an image. For security purposes you don't need a full video, just a good resolution picture of a face or license plate. So we have ours set to capture an image once per second and hold onto it for 60 days. We have about 30 cameras feeding into this system and it only has 2 terabyte of space in the hard drives, which we've never filled up.
|
|
|
|
|
Save it on removable harddrives, then sneakernet it to a secure location.
For that quantity of data, you probably have to go to eSATA, as there probably isn't much else that will be fast enough. There are (or at least, were) a few enclosure systems that supported USB 2, eSATA and had a custom bay so they plugged into the system like a jumbo floppy. The USB2 would likely be convenient to provide offline access without requiring special workstation setups.
Rather then try to handle it like a scheduled backup task, I'd set the system up to continually mirror the data onto the removable drives, then once a week swap them out. That way, the data path probably won't be a bottleneck.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
|
|
|
|
|
I used to use a program called GOTCHA! that would only record changes.
So if you filmed yourself raising your hand and holding there for ten minutes before lowering it, on playback you'd just see you raise your hand and immediately drop it.
For interior monitoring it was great. Outside not as much because winds and shifting shadows would trigger it. But you could mask the screen to mark areas to ignore (like bushes) and areas to watch (like sidewalks)
That could reduce the recordings quite a bit. Changed images are timestamped so you can see when motion occurred.
I used it once to monitor a whiteboard that some chucklehead would alter my diagrams. I wanted to catch who was trying to sabotage my system designs. Instead, I caught a monthly cleaning crew having their way with my office and walking off with several hundred dollars of equipment.
Psychosis at 10
Film at 11
Those who do not remember the past, are doomed to repeat it.
Those who do not remember the past, cannot build upon it.
|
|
|
|
|
|
Yes. It's getting a little long in the tooth and the video format is non-standard, but it can get the job done.
Psychosis at 10
Film at 11
Those who do not remember the past, are doomed to repeat it.
Those who do not remember the past, cannot build upon it.
|
|
|
|
|
I have read a lot of the comments. Here are my 2 cents, having dealt with a lot of data, and some of the rules of imaging/cameras.
1) I definitely prefer hard wired cameras, WiFi can easily be attacked, and attacked locally.
2) Think bandwidth, you should have Cat 5 going into Gigabit or better switches
3) Certainly, every camera should be encrypting their output.
4) Because of hiccups with the OC3/External connection, you should have the ability to store some HOURS of video locally. Preferably 3-7 days if possible.
5) Consider only uploading extremely compressed feeds. 5 to 10 fps vs. 30fps.
6) Design for failure. When your RAID Degrades, it has to NOT become readonly! But transfer times will drop. Too many devices writing to one device will clog the pipe.
We had to redesign a system from ONE server to 3 servers because the bandwidth of the incoming and outgoing network cards could never be maintained. 2 servers worked, but no redundancy, and once we had to redesign things to talk to different servers, we took the extra step. The project grew a bit, and the extra server helped.
7) Plan for growth. I don't care that the number of cameras as never changed. If you can't handle a few more cameras, you are too close to an edge. 10% more cameras would make me feel safe.
A friend implemented a local storage of video, and external storage of a "snapshot" picture every 3 seconds. It was very very efficient. Some of the snapshots increased if there was motion.
So, there are a lot of variables. The big question is always WHY?
WHY do they want to keep the video? Do they need it for police, investigations, etc? Do they need to just prove somebody, or do they need to ID the person. The whys involved here should determine what approaches you can and should use. Many cameras can be lower quality, some will be required to pickup a license plate or other details, and again, using techniques that change the quality as activity is detected really help to "compress" the actual data.
Tons of questions.
Good luck.
|
|
|
|
|
I have a professional home security camera system, 16 High Res Cameras. The 1 T disk can store for about 2 weeks. They store ONLY WHEN THEY SEE MOTION!@
Take a look at the off the shelf equipment at Security Camera Direct.
Wire it!
The Irishman
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
