Click here to Skip to main content
15,889,931 members
Home / Discussions / Design and Architecture
   

Design and Architecture

 
GeneralRe: Design pattern video tutorials Pin
DrSinghnri19-May-17 20:39
DrSinghnri19-May-17 20:39 
AnswerRe: Design pattern video tutorials Pin
Pete O'Hanlon8-May-17 23:03
mvePete O'Hanlon8-May-17 23:03 
QuestionAn architectural sign ! Pin
Foxxx F5-May-17 22:00
Foxxx F5-May-17 22:00 
AnswerRe: An architectural sign ! Pin
Gerry Schmitz6-May-17 2:34
mveGerry Schmitz6-May-17 2:34 
AnswerRe: An architectural sign ! Pin
Eddy Vluggen6-May-17 3:30
professionalEddy Vluggen6-May-17 3:30 
AnswerRe: An architectural sign ! Pin
Mycroft Holmes6-May-17 14:04
professionalMycroft Holmes6-May-17 14:04 
QuestionSimple N-Tier design Pin
Danpeking18-Apr-17 4:39
Danpeking18-Apr-17 4:39 
QuestionDesign a opportunistic exists-in-set query Pin
Member 1247514217-Apr-17 17:33
Member 1247514217-Apr-17 17:33 
I need to design a system where I need to check if a key exists or not.
Currently I am already using aerospike for the cause but my use case is slightly different.

I will have a key like this ( A group of 3 integers)
<clientid>-<campaignid>-<userid>

For eg.
1-105-1133441
222-1000-1891893



I just need to know if this key exists or not in the set.
If exists return 1 else set the key and return 0. So that next time the query will return 1

This has to be a tcp service because the checks/set will happen from multiple servers

The point is if a campaign is live the queries will be very dense ( upto 20k/s ). So the key lookup has to be in memory. But after that I still have to retain the set but it has to be on disk , so that memory can be freed up for other campaigns. I do not expect more that 20-30 campaigns to be live.

Whenever the next time an old campaign gets a query again , the server should bring it back to memory and serve the requests from memory. The first query will be slow because it has to kickoff a read from disk but next query onwards will be very fast.

I am using aerospike for now , but there is a problem that aeorspike stores all keys in memory. When it is practically useless to have old campaigns in memory unless they get triggered again

The total number of keys is practially unlimited. But let us consider 50 million keys per campaign and I may need to store 100k campaign data. Of which only < 50 will be live at any given point of time.


Any ideas how I can implement this

So what should I use for this
AnswerRe: Design a opportunistic exists-in-set query Pin
Gerry Schmitz18-Apr-17 6:09
mveGerry Schmitz18-Apr-17 6:09 
QuestionNeed 2 auto-generation puzzle programs created Pin
JustifiedPub30-Mar-17 8:54
JustifiedPub30-Mar-17 8:54 
AnswerRe: Need 2 auto-generation puzzle programs created Pin
Pete O'Hanlon30-Mar-17 9:10
mvePete O'Hanlon30-Mar-17 9:10 
AnswerRe: Need 2 auto-generation puzzle programs created Pin
ZurdoDev30-Mar-17 10:10
professionalZurdoDev30-Mar-17 10:10 
QuestionMake a general makefile at root to search subdirectories and pick the path variables Pin
Jitesh Varma15-Mar-17 3:52
professionalJitesh Varma15-Mar-17 3:52 
RantRe: Make a general makefile at root to search subdirectories and pick the path variables Pin
CHill6015-Mar-17 5:42
mveCHill6015-Mar-17 5:42 
GeneralRe: Make a general makefile at root to search subdirectories and pick the path variables Pin
Jitesh Varma15-Mar-17 6:57
professionalJitesh Varma15-Mar-17 6:57 
GeneralRe: Make a general makefile at root to search subdirectories and pick the path variables Pin
CHill6015-Mar-17 13:55
mveCHill6015-Mar-17 13:55 
GeneralRe: Make a general makefile at root to search subdirectories and pick the path variables Pin
Jitesh Varma16-Mar-17 3:09
professionalJitesh Varma16-Mar-17 3:09 
QuestionTFS DownloadFiles activity works slowly Pin
andreybedny1-Mar-17 22:44
andreybedny1-Mar-17 22:44 
QuestionCREATING A TABLE Pin
Member 1300689616-Feb-17 21:57
Member 1300689616-Feb-17 21:57 
AnswerRe: CREATING A TABLE Pin
Richard Andrew x6421-Feb-17 7:16
professionalRichard Andrew x6421-Feb-17 7:16 
AnswerRe: CREATING A TABLE Pin
Munchies_Matt4-Mar-17 7:39
Munchies_Matt4-Mar-17 7:39 
AnswerRe: CREATING A TABLE Pin
Gerry Schmitz6-Mar-17 10:04
mveGerry Schmitz6-Mar-17 10:04 
AnswerRe: CREATING A TABLE Pin
ZurdoDev15-Mar-17 6:24
professionalZurdoDev15-Mar-17 6:24 
QuestionUnity in Modular App Pin
aboubkr9015-Feb-17 2:24
professionalaboubkr9015-Feb-17 2:24 
AnswerRe: Unity in Modular App Pin
Pete O'Hanlon15-Feb-17 2:27
mvePete O'Hanlon15-Feb-17 2:27 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.