|
In my application I manage multiple resources, i.e. potentially large binary files (i.e. images).
I want each image to be added only once. But I try to make this comparison quick instead of comparing each bytes of the new image/resource to each bytes of existing resources.
The trick I came up with is to compute a unique ID for each resource using MD5. Which nicely return a unique 16 bytes array, which nicely convert into a Guid, which I can then nicely store into a Dictionary<Guid, Resource>
Now it was brought to my attention that MDA5 is "insecure" and I should use SHA256 instead.
The problem I have with that (hence my question) is that SHA256 is more annoying to use, my data should be a multiple of 16 bytes long. Also it return a 32 byte array which cannot easily be turned into Guid.
So here is my conundrum. All I want is a unique ID for resource. This is not a security risk, just a quicker way to avoid resource duplication inside the document.
Do I really need to use SHA256? i.e. is it likely the "insecure nature" of MD5 will accidentally collide hash keys of random resources / images?
|
|
|
|
|
So, MD5 is insecure. Who cares? You're not "securing" anything with it. You're just generating a relatively unique ID for an image.
ANY hash can generate collisions. After all, it's impossible to represent every possible stream of bytes (which is theoretically infinite) in a finite number of bits. The shorter the resulting hash value, the higher the chance of a collision. The hash value of MD5 is short compared to most hashing algorithms, so it's going to have a higher chance to generate a collision.
If you want to calculate the chance yourself, see Hash Collision Probabilities[^]. It comes down to the number of images you're going to be handling.
modified 14-Nov-18 22:51pm.
|
|
|
|
|
Nice link! 
I guess if I am really chicken can compare the byte array if the hash is identical...
But yeah, I will go with who cares! thanks for motivation!
|
|
|
|
|
MD5 is officially "broken" - which means that it is possible under some circumstances to regenerate the original input from the MD5 hash. That's bad, if you are storing passwords.
But ... you aren't. You are using this for a "quick comparison" function where you really can't regenerate the original document from the hash value (because the document size is too big) and it wouldn't matter if you could!
Go for it - use MD5 by all means, and comment your decision so that the idiot who comes after you can't bad mouth you for "security reasons". He'll find other reasons anyway: Obligatory Dilbert[^]
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
very clearly articulated rationale!
thanks!
|
|
|
|
|
You're welcome!
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
I have a program that uses the WebBrowser to scrape a website. As each page is scraped for data, I call event handlers to write the data. I need to store the AccountId for unique pricing in the database.
I really don't want to modify the eventHandlers called "WebBrowserDocumentCompletedEventArgs" because I'm adding and removing them so often. I tried a global class but the event handler is not picking it up.
I'm looking for a durable way to store this single string, it's a MongoDB Id.
I suppose I could store it in the registry? or is that a bad idea.
Store it in a Mongo document? Does sound better.
I don't need the Id till I do the final batch write.
If it ain't broke don't fix it
Discover my world at jkirkerx.com
|
|
|
|
|
Registry? Bad idea. Don't store anything in the registry. It's restricted access now, because people stored everything in the registry so it became a bloated mess. It's likely to become more restricted in the future, not less.
Store it anywhere else: settings file, DB, Excel file, Inca Quipu/Khipu Knotted ropes.
Anywhere but the registry.
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
I'll have to create a new method of storing stuff then. Right now I store the screen size, screen location in the registry. But your right, find another place.
If it ain't broke don't fix it
Discover my world at jkirkerx.com
|
|
|
|
|
I think it depends on what the final batch is going to look like.
If you're associating data, how you persist it depends greatly on what you ultimately want to do with it. If you're potentially associating a piece of data with multiple other pieces of data, or if you're going to need complex querying, a database is most likely the right fit.
If you're building a fairly flat data association, a file is just fine, and the question shifts to how you intend that data be consumed.
Are you providing it through RSS? Use XML serialization. Through a web service? Use JSON serialization. Human readable? Wrtielines to a txt. Making a pretty report for management? Put yourself through the living hell that is iTextSharp and generate PDFs.
"Never attribute to malice that which can be explained by stupidity."
- Hanlon's Razor
|
|
|
|
|
It's just unique pricing from a channel distributor for that chosen customer account.
Manufacture > Products > Pricing
string Id
string AccountId
decimal MSRP
decimal Price
If it ain't broke don't fix it
Discover my world at jkirkerx.com
|
|
|
|
|
jkirkerx wrote: I really don't want to modify the eventHandlers called "WebBrowserDocumentCompletedEventArgs" because I'm adding and removing them so often.
I think if we know why you are adding/removing frequently, that will, perhaps, lead to more insight into the issue.
«Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?» T. S. Elliot
|
|
|
|
|
I'm using the Web Browser control. Each Time I change the URL, I add an event handler called Document Complete. When it fires I unload the handler and scrape the page data. Then I go to the next page and reload the event handler.
Some pages are more complex, so I call the Url, load the event handler Document Complete, then unload the handler and go through all the links on the parent page, and call those Urls while calling a child version of Document Complete. Other pages have a Json file that I can just download and scrape the data so I call a JSON version of Document Complete.
So depending on what I detect on the page, I call the appropriate version of Document Complete.
The reason why I need to store the AccountId is for when I write the pricing, that is unique for each account.
Sounds silly, why can't I just go straight to the cloud database for this. Because I can grab everything, the product, it's images, associated videos and pdf's, and group them together into the database. Then I can generate emails that showcase the product with all the needed resources to promote it. Or create Excel spreadsheets with the image, pricing matrix and delivery dates.
If it ain't broke don't fix it
Discover my world at jkirkerx.com
|
|
|
|
|
|
I'll try it since you wrote it!
wow 10 years old now. That was the shameless plug for your project.
If it ain't broke don't fix it
Discover my world at jkirkerx.com
|
|
|
|
|
Hi,
for example i have to different charts (A and B). all charts have the same tracks (but their value may be different and only one value of them is unique but not equal). Imagine two drilled hole beside each other one has a more attitude than the other. I have divided of each thickness of holes to :1A,1B-2A,2B and so on. now i want to relate (bound) these same sections of two different charts together with drawing lines that pass through two charts. can anybody coach me?
thank you in advance
modified 14-Nov-18 18:16pm.
|
|
|
|
|
Is it possible to detect if an app us running on our network? or VPN'd in? If so, how?
Thanks
If it's not broken, fix it until it is.
Everything makes sense in someone's mind.
Ya can't fix stupid.
|
|
|
|
|
Is there a webserver running on the network? If not, grab a raspberry Pi and make one.
Try downloading a text-file from that webserver over the local LAN-address. If it works, you're either on your own network, or someone is trying to convince your app that it is.
Alternatively, you ask one of the network-admins for some help; I'll bet there's a lot of things that identify your network as yours when running netstat, ipconfig, whois and nslookup
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
That depends, and there are a few different approaches depending on your network configuration. It also depends on how much say you have in terms of how change is affected.
If you are using a Domain and the network is configured only to allow domain machines (via 802.1x or some other standard), then you can query WMI on windows machines on the domain using a privileged account, generally using powershell or a custom tool. You can mine running processes through WMI, and I would classify this as the "best" way to find out if a bit of software is running. Linux machines are easier, the BASH command ps-A will give you a running process list. You could do the same with local admin accounts, of course, but that approach really doesn't scale well.
Unfortunately, because of the pretty extreme strictures of Frame-level security, most networks do not have a domain lock on their network. You can run a port scan on a computer to try to determine what responsive applications are running, but that's not terribly consistent, especially with the number of applications that will use ephemeral ports, and can only determine applications that will handle in-bound connections.
You could require a connection client exist on computers before they get a DHCP lease; Cisco AnyConnect has modules that can do this, and I'm sure there's other pre-baked solutions for that. That connection client can have compliance portions that can do just about anything on the host, like process or file enumeration, as well as AV definition version checks, and so on. This is a medium-good solution, since anyone that can figure out how your switches are subnetted can just set a static IP.
Network traffic analysis will generally reveal applications that are communicating over the network with a fair degree of reliability. Best practices would say that you run a network proxy for hosts inside your domain, otherwise you'll have a bunch of encrypted traffic passing in and out of your network with no visibility into what that traffic is; not great for preventing intrusion or exfiltration. With un-encrypted traffic, a number of tools can fingerprint traffic automatically. From a pragmatic operational standpoint, this is generally the most feasible option for application fingerprinting.
Same answers apply to VPN, bearing in mind that the VPN address range should be subnetted differently than native internal systems.
"Never attribute to malice that which can be explained by stupidity."
- Hanlon's Razor
|
|
|
|
|
This is a somewhat wide subject, so the QnA was inappropriate.
I am working on some games and apps, and they need to talk to some MySQL servers and/or Azure resources, and so I need some connection strings. Usually I've made internal tools, where having the connection strings in code or in a XML or Json -file, was unproblematic.
Now I'm having the issue that APK's are basically Zip-files, and .net executables are de-compilable.
My first solution is to setup an intermediary web service to keep the connection strings unavailable from the app.
But I can't help but think that I'm missing something here.
Also, an extra middle-man webserver, increase the possible bottle-necks and failure points
What are this community's thoughts on the subject?
Thanks for your time!
|
|
|
|
|
Frank R. Haugen wrote: This is a somewhat wide subject, so the QnA was inappropriate.
Actually quite the opposite. More people would see this in Q&A, rather than just those who look at C# questions (which this is not).
|
|
|
|
|
Frank R. Haugen wrote: Now I'm having the issue that APK's are basically Zip-files, and .net executables are de-compilable.
Does not seem an issue for Rimworld.
Frank R. Haugen wrote: What are this community's thoughts on the subject?
If you can't afford the user in the database, don't give away the connectionstring. Simple as that. Third parties can't keep secrets on a computer that isn't theirs.
Who is going to be the "owner" of the data? You, or the user? Is the user going to be allowed to make changes? If no, simply give them a connection-string that's hooked to a user with limited (read) access.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
Hello there. I want to do an ftp server-client program with c #. I created free hostting for this, but this hostting does not give me ftp authorization, can you help me with this.
|
|
|
|
|
We can't add FTP authorization for you - you need to talk to your hosting service and see what they can do for you. It may be that your free plan does not include any FTP access and may well be blocked by their firewall (free plans are generally limited in some ways, and I don't bother with them at all any more as they have all been more hassle than they were worth) but that is something you will have to discuss with your provider.
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Free hosting companies make their money by showing ads on the sites that you host there. Abusing their service to provide drive-functionality in the cloud would be probably in violation of their terms of use.
..but that's why they block FTP. It's a question that comes by a few times a year. If you want FTP, you will probably have to pay for the service.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
