|
Quote: The quantity I'm considering would not work with on-site storage; this quantity would have to be off-site.
Why?
What is the use case for the data? If it is purely for security, and will only be reviewed after an incident, then majority of it will be written, archived for a week, and finally deleted. Why set up a system to move all that data around? Keep it as close to the camera as possible.
I'd look at a node consisting of simple PC, redundant storage, recording from several cameras, connected to dedicated security (preferably wired) network and accessible from central control station. Easier to add more cameras too.
When access to the video is required, simply copy relevant files based on incident time to a central location.
|
|
|
|
|
NeverJustHere wrote: (preferably wired)
How come ?
Just for reference, in this case at least two different humans will be viewing the same camera, and as many as 25 at one time.
|
|
|
|
|
If someone figures out the security system is dependent on WiFi (or buys the details from a disgruntled ex-employee), then they can jam frequencies or perform denial of service attacks to mess with the system. Wired would not have this problem.
|
|
|
|
|
At some point this could almost be easier and cheaper if you just hired some human to watch the camera and take appropriate action if whatever happens happens...
|
|
|
|
|
I don't where your job is or what it is but does your project have any legal requirement for storage? Where I used to work we had to keep video of all cameras for the last 3 weeks at a minimum and system redundancy.
Due to compression you're not going to see 500GB from one camera. Like has already been said, I recommend testing and monitoring using a busy location and using local storage for a couple of weeks to get a good data set and bandwidth pattern over time.
Depending on requirements and the hardware chosen you may even be able get away with adjusting compression ratios, increasing compression for dormant locations or those not as security sensitive and lowering it for higher traffic/security areas. This won't get you a lot more room to play with but if you run up against limitations, it may help you.
Also, having an off-site storage is problematic for that many cameras. Sure you can use an OC3 (probably going to need it!) but it's expensive as hell and it's a single point of failure. If the line gets cut all your cameras go down.
If off-site is a mandatory requirement, you'd be better served having multiple connections from multiple vendors going to off-site storage. Groups of cameras would use each connection. In that case, if one vendor goes down or connection gets cut, you don't lose all of your camera coverage.
I wouldn't go with an all-in-one system that has 500 camera feeds to it. I'd go with a much smaller system that handles, say, about 50 cameras at once and has it's own storage system. Then you have at least 10 of these systems spread throughout your site(s) making your system modular, expandable and fault tolerant. You can then have a "supervisor" system that manages them and this also gives you the opportunity to start with a smaller implementation and implement it in stages. You also get the time to find and fix bugs along the way before you get to your full implementation.
|
|
|
|
|
Dave Kreskowiak wrote: does your project have any legal requirement for storage?
Now there's a question I didn't think to ask.
I never thought the government would be interested in a vid clip of cars in a parking lot.
What is the typical name of the government agency who gets involved in such requirements ?
Dave Kreskowiak wrote: If off-site is a mandatory requirement,
Thanks for the extra brain cells. I think you guys have convinced me to run a test first and see what kind of data numbers I'm likely to generate.
The off-site thing is a result of me thinking: the humans who will be using the video are definitely not going to be the nerd type. My thoughts are/were that normal humans these days want a website with a simple "click here to view yesterday's tape" sort of thing
|
|
|
|
|
C-P-User-3 wrote: What is the typical name of the government agency who gets involved in such requirements ?
Speaking of taking video of cars in a parking lot, we call that a border crossing.  The Department of Homeland Security, U.S. Customs and Border Protection.
|
|
|
|
|
Until they finally tore up the toll booths - Highway 400 in Atlanta 
Airport parking too...
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
C-P-User-3 wrote: What is the typical name of the government agency who gets involved in such requirements ?
The Department of Administrative Affairs?
If you have an important point to make, don't try to be subtle or clever. Use a pile driver. Hit the point once. Then come back and hit it again. Then hit it a third time - a tremendous whack.
--Winston Churchill
|
|
|
|
|
I have a very interesting project about the same line...
First of all the storage not that huge as you estimated as there is compression in the middle too!
In the design we thought to be the final we paired every endpoint with a single board computer (like Raspberry Pi), with a smart software of its own...
So the camera actually stores data on the local single board computer and those computers are connecting to a server farm (with fast and expensive but not too large storage) via a interface (API) that enables them to upload content...
Day-by-day you will be a bit offline, but you can design your system with an emergency line, that enables to some of the cameras to go online...
The data from the server farm downloaded to some backup servers (slower and cheaper) to clear the fast storage for daily work...
Do these things:
1. Group you cameras by importance
2. Sample each group to estimate storage need
3. Design your system's parts
Skipper: We'll fix it.
Alex: Fix it? How you gonna fix this?
Skipper: Grit, spit and a whole lotta duct tape.
|
|
|
|
|
I've briefly worked on such a system. Some include motion sensors along with the camera, while others analyze the video data. As long as no motion is detected, they omit the data from long-term storage, or only store a brief sample (one frame per second/minute, etc.). That, along with conventional video compression, reduces the storage requirements from outlandish to merely huge.
Software Zen: delete this;
|
|
|
|
|
You can also configure most cameras to use 'Motion-Detection' setting. In my experience, most cameras are installed in places where pictures very rarely change.
Behzad
|
|
|
|
|
The system that I've had experience allows you to set how often it captures an image. For security purposes you don't need a full video, just a good resolution picture of a face or license plate. So we have ours set to capture an image once per second and hold onto it for 60 days. We have about 30 cameras feeding into this system and it only has 2 terabyte of space in the hard drives, which we've never filled up.
|
|
|
|
|
Save it on removable harddrives, then sneakernet it to a secure location.
For that quantity of data, you probably have to go to eSATA, as there probably isn't much else that will be fast enough. There are (or at least, were) a few enclosure systems that supported USB 2, eSATA and had a custom bay so they plugged into the system like a jumbo floppy. The USB2 would likely be convenient to provide offline access without requiring special workstation setups.
Rather then try to handle it like a scheduled backup task, I'd set the system up to continually mirror the data onto the removable drives, then once a week swap them out. That way, the data path probably won't be a bottleneck.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
|
|
|
|
|
I used to use a program called GOTCHA! that would only record changes.
So if you filmed yourself raising your hand and holding there for ten minutes before lowering it, on playback you'd just see you raise your hand and immediately drop it.
For interior monitoring it was great. Outside not as much because winds and shifting shadows would trigger it. But you could mask the screen to mark areas to ignore (like bushes) and areas to watch (like sidewalks)
That could reduce the recordings quite a bit. Changed images are timestamped so you can see when motion occurred.
I used it once to monitor a whiteboard that some chucklehead would alter my diagrams. I wanted to catch who was trying to sabotage my system designs. Instead, I caught a monthly cleaning crew having their way with my office and walking off with several hundred dollars of equipment.
Psychosis at 10
Film at 11
Those who do not remember the past, are doomed to repeat it.
Those who do not remember the past, cannot build upon it.
|
|
|
|
|
|
Yes. It's getting a little long in the tooth and the video format is non-standard, but it can get the job done.
Psychosis at 10
Film at 11
Those who do not remember the past, are doomed to repeat it.
Those who do not remember the past, cannot build upon it.
|
|
|
|
|
I have read a lot of the comments. Here are my 2 cents, having dealt with a lot of data, and some of the rules of imaging/cameras.
1) I definitely prefer hard wired cameras, WiFi can easily be attacked, and attacked locally.
2) Think bandwidth, you should have Cat 5 going into Gigabit or better switches
3) Certainly, every camera should be encrypting their output.
4) Because of hiccups with the OC3/External connection, you should have the ability to store some HOURS of video locally. Preferably 3-7 days if possible.
5) Consider only uploading extremely compressed feeds. 5 to 10 fps vs. 30fps.
6) Design for failure. When your RAID Degrades, it has to NOT become readonly! But transfer times will drop. Too many devices writing to one device will clog the pipe.
We had to redesign a system from ONE server to 3 servers because the bandwidth of the incoming and outgoing network cards could never be maintained. 2 servers worked, but no redundancy, and once we had to redesign things to talk to different servers, we took the extra step. The project grew a bit, and the extra server helped.
7) Plan for growth. I don't care that the number of cameras as never changed. If you can't handle a few more cameras, you are too close to an edge. 10% more cameras would make me feel safe.
A friend implemented a local storage of video, and external storage of a "snapshot" picture every 3 seconds. It was very very efficient. Some of the snapshots increased if there was motion.
So, there are a lot of variables. The big question is always WHY?
WHY do they want to keep the video? Do they need it for police, investigations, etc? Do they need to just prove somebody, or do they need to ID the person. The whys involved here should determine what approaches you can and should use. Many cameras can be lower quality, some will be required to pickup a license plate or other details, and again, using techniques that change the quality as activity is detected really help to "compress" the actual data.
Tons of questions.
Good luck.
|
|
|
|
|
I have a professional home security camera system, 16 High Res Cameras. The 1 T disk can store for about 2 weeks. They store ONLY WHEN THEY SEE MOTION!@
Take a look at the off the shelf equipment at Security Camera Direct.
Wire it!
The Irishman
|
|
|
|
|
|
Yes
I've been using their 16 camera, High res with IR for about 4 years
There very good with any help
Just be sure to get any adapters for the cable 12 volt power to the camera
The Irishman
|
|
|
|
|
Kirk 10389821 wrote: We had to redesign a system from ONE server to 3 servers because the bandwidth of the incoming and outgoing network cards could never be maintained. 2 servers worked, but no redundancy, and once we had to redesign things to talk to different servers, we took the extra step.
I totally missed the concept you are trying to describe. More explanation is welcome.
Kirk 10389821 wrote: 7) Plan for growth.
Smirk ! Mind reader ??? Kreskin ? If not, you and I are definitely parallel thinkers.
That's a key pillar of the business concept.
From what we're seeing in the logistics of software, hardware, and firmware, plus the connections (an internet connection is integral to this) that's also the single biggest problem to overcome. In fact, we're probably spending more time on this one problem (adding more cameras) than all the others, as we definitely want the customer to do exactly that.
Kirk 10389821 wrote: The big question is always WHY?
Nice brain. We in the gang here will have a philosophical round table about those sorts of concepts you've described. Thanks for the input; on all of your post.
I will upvote it.
modified 13-Jul-15 16:10pm.
|
|
|
|
|
C-P-User-3 wrote: I totally missed the concept you are trying to describe. More explanation is welcome.
There was a life-lesson in here. The design had 100Mb incoming network card, and a 100Mb outgoing network card. Little did we know that the hard drive and the bus could not keep them busy. (This was circa 1998). We had incoming files coming through at full throttle. But when we turned on the outbound channel, it was maybe doing 20-25% of 100Mb. It took a lot of math and analysis to see we had the server not only reading but writing to the HD, and then reading other areas and trying to send it out via the network. Between the drive max throughput, and the BUS speed, we had over-estimated what we could accomplish. And testing it by testing only the incoming half and the outgoing half (separately) was not good enough to find the problem. (That seems obvious right now, and the life lesson is test it like it is ACTUALLY going to be used).
C-P-User-3 wrote: nice brain. We in the gang here will have a philosophical round table about those sorts of concepts you've described. Thanks for the input; on all of your post.
Thanks... I hope it helps. For me, the WHY is the most important question. It is what allows you to solve problems in the most interesting ways. I often remind new clients that my job isn't to give them what they want, but to deliver what they need! That comes from getting into the why! As does knowing how to design the system properly in the first place. (Actually, we audited a compound that had video cameras with labels indicating the IP address of the camera. These were security cameras, at least they were supposed to be. I have no problem with labeling the equipment with Random TAG numbers that require a secure spreadsheet to look them up.)
|
|
|
|
|
This is a little crazy. How about recording only when motion is detected? That alone would save tons of resources, at the expense of a little complexity.
|
|
|
|
|
A pipe just burst in the wall in my mother's bathroom. We have called the water department, and they have a guy on his way right now, having told him to go into 'emergency mode'.
What a pain, and the day after my father's funeral as well.
I hate life. I would change it, but I don't have access to the source code. (With regards to the member who has something along those lines in his sig).
What do you get when you cross a joke with a rhetorical question?
The metaphorical solid rear-end expulsions have impacted the metaphorical motorized bladed rotating air movement mechanism.
Do questions with multiple question marks annoy you???
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
