Click here to Skip to main content
15,849,180 members
Articles / Artificial Intelligence
Article

Package Delivery Detection using CodeProject.AI Server and Blue Iris

Rate me:
Please Sign up or sign in to vote.
5.00/5 (4 votes)
9 Dec 2022CPOL6 min read 17.1K   8   17
In this article, I download a custom model to detect packages, set up Blue Iris to detect the package, trigger an alert, and send an email notification that a package has arrived.
This article demonstrates how to use CodeProject.AI Server and Blue Iris to detect delivery packages using object recognition and trigger email notifications for alerts.

So far in my previous articles, we've been doing a lot of setting things up. In this article, I'd like to take what we've learned so far and use CodeProject.AI Server and Blue Iris for something practical, like detecting a delivery package that triggers an alert to send an email notification. Let's dive in.

Getting the Package Custom Model

If you haven't yet set up Blue Iris and CodeProject.AI Server, please check out my article, How to Setup Blue Iris and CodeProject.AI Server on Windows Using a Wyze Cam.

First, we need to get a model that detects packages. The great Mike Lud, CodeProject Community Engineer is training many models and has developed a model that detects packages. So the first step is to download the package model, and put it in the custom model folder. Go to Mike Lud's GitHub, and download package.pt. Then, copy package.pt into the custom model folder for CodeProject.AI Server, which is C:\Program Files\CodeProject\AI\modules\ObjectDetectionYolo\custom-models.

Setting up Blue Iris Alerts

Image 1

Now, launch Blue Iris. The next step is to set up the email server. Go to the Blue Iris main settings, then the Emails servers tab. Hit the + symbol to enter the email settings.

Image 2

Enter a description for the email profile. In my case, I have a throwaway Outlook account I sometimes use, so my Description is "Outlook Stevan". From the Preset dropdown, choose "Windows Live Hotmail." There are multiple options here, including AOL Mail, Gmail, Yahoo, and Zoho. Choosing one of the presets automatically fills in the Server address. You can also choose User defined, but you'll need to know the server address in that case.

Next, input the From email (I use the same email to which the alert is going), and enter the Login email (which is the email you wish to receive the notification), and the password for that email account. Then hit Test.

Image 3

Perfect! Email notifications are working. Hit OK to exit the email profile, then OK again to exit the Blue Iris settings.

Image 4

Now it's time to set up the alerts. Go to the camera settings, and the Alerts tab. I actually left all the main settings in the alerts menu default. From here, click On Alert.

Image 5

Then click the + icon to add a new alert. A dropdown loads. From there, select Send email.

Image 6

In the Required AI objects field, input "package". Then make sure the Outgoing (SMTP) server has the profile selected. In my case, "Outlook Stevan."

In the Recipient(s) field, input the email address that will receive the alert. Then make the subject and body of the email whatever you wish. Hit OK to exit the email configuration alert settings, OK to exit the Action set menu, and OK to accept the camera settings.

All right, let's detect some packages! My wife doesn't want the camera setup at the door so it's just in my office. But thankfully, I have some packages to test.

Image 7

From the AI tab in the Blue Iris settings, click Open AI control panel to launch CodeProject.AI Server. Also make sure that the Use custom model folder box is checked, since that's where packages.pt is located.

Image 8

Next, go to the Trigger tab in the camera settings, then hit Artificial Intelligence.

Make sure To confirm says "package" and Custom models also says "package". Hit OK to accept the Artificial Intelligence settings, then OK to accept the camera settings.

Detecting a Package - Part 1

Image 9

From CodeProject.AI Server, we don't really need anything other than Object Detection (YOLO), so disable everything else by clicking on the three dots next to the process and picking Stop from the dropdown menu.

Image 10

The camera is on the floor because I couldn't get it higher without mounting it or putting the camera upside down. I'm trying to simulate a front door, so this should be good enough. Time to place a package in the camera's view!

Image 11

I took a package and placed it the camera's view. And that's it! Package detected!

Now to check Outlook to see if an email alert was sent.

Image 12

There's the test email, and the email alert! Awesome!

One final test. Let's huck another package in there, and see if it can detect a second package.

An Error Sending Email Notifications

This is where I experienced some troubles. For awhile, putting various packages in front of the camera did trigger an alert. But it was sporadic. Part of the problem was my package. The first package detected was a box. The box was pretty reliably detected every time. The next package I used was a large envelope. The envelope was detected less frequently. In addition, even if I could see the package was detected in Blue Iris, and confirm an alert was triggered, the email notifications stopped working. So I went back to the Email Servers tab in the main Blue Iris settings and hit the Test button again. I got an error:

Error with sending data - data confirmation; Server says: 554 5.2.0
STOREDRV.Submission.Exception:OutboundSpamException; Failed to process message due to a
permanent exception with message [BeginDiagnosticData]WASCL UserAction verdict is not
None. Actual verdict is RefuseQuota, ShowTierUpgrade. OutboundSpamException: WASCL
UserAction verdict is not None. Actual verdict is RefuseQuota, ShowTierUpgrade.
[EndDiagnosticData] [Hostname=FR1P15201MB2167.LAMP152.PROD.OUTLOOK.COM]

Image 13

I asked for some help understanding the error on the ipcamtalk forum and apparently email service providers have email sending limits. This article details more specifics, but what likely happened in my case is that because I was using an unverified, throwaway Outlook account that I barely used, the emails were no longer sending because I hit my limit for number of emails sent.

Detecting a Package - Part 2

So I decided to setup a Gmail account and see if that helped. I created a new email profile by hitting the + symbol, and followed the same process as before. The only difference is that Gmail requires Oauth authentication rather than a password.

Image 14

Click the Sign in with Google button, then follow the instructions.

Now to start over and put a package in front of the camera, and see if Gmail detects it.

Image 15

Success! Package detected, alert triggered, and Gmail notification sent.

One final test (for real this time). Let's huck another package in the camera view and see if the second package is detected. It's the holiday season and the overworked Amazon employees will take any shortcut they can.

Image 16

Huck!

Image 17

Bingo! The second package was detected, an alert was triggered, and a Gmail notification was sent. I believe I have successfully set up CodeProject.AI Server to detect delivery packages using Blue Iris.

If you should have any questions or problems with your setup, please feel free to ask them in the article forum below.

This article is part of the series 'CodeProject.AI View All

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Technical Writer CodeProject
Canada Canada
Sean Ewington is the Content Manager for CodeProject.

His background in programming is primarily C++ and HTML, but has experience in other, "unsavoury" languages.

He loves movies, and likes to say inconceivable often, even if it does not mean what he thinks it means.

Comments and Discussions

 
QuestionPackage Object Recognition Training Pin
C3dC0dgy21-Nov-23 9:32
C3dC0dgy21-Nov-23 9:32 
AnswerRe: Package Object Recognition Training Pin
Sean Ewington23-Nov-23 11:03
staffSean Ewington23-Nov-23 11:03 
GeneralRe: Package Object Recognition Training Pin
autumnwalker24-Jan-24 8:09
autumnwalker24-Jan-24 8:09 
GeneralRe: Package Object Recognition Training Pin
Sean Ewington26-Jan-24 8:45
staffSean Ewington26-Jan-24 8:45 
GeneralRe: Package Object Recognition Training Pin
autumnwalker29-Jan-24 2:39
autumnwalker29-Jan-24 2:39 
GeneralRe: Package Object Recognition Training Pin
Sean Ewington28-Feb-24 11:06
staffSean Ewington28-Feb-24 11:06 
QuestionPackage alert precedence Pin
KittenMcnuggets17-Jul-23 16:11
KittenMcnuggets17-Jul-23 16:11 
QuestionHow to add using Codeproject in docker container? Pin
Member 1598926226-Apr-23 7:30
Member 1598926226-Apr-23 7:30 
AnswerRe: How to add using Codeproject in docker container? Pin
Sean Ewington26-Apr-23 7:33
staffSean Ewington26-Apr-23 7:33 
GeneralRe: How to add using Codeproject in docker container? Pin
Member 1598926226-Apr-23 18:32
Member 1598926226-Apr-23 18:32 
QuestionPackage.PT include with Sense AI? Pin
coolspot1819-Feb-23 4:56
coolspot1819-Feb-23 4:56 
QuestionDifferent package model? Pin
Member 932665612-Jan-23 7:44
Member 932665612-Jan-23 7:44 
AnswerRe: Different package model? Pin
Sean Ewington20-Jan-23 5:56
staffSean Ewington20-Jan-23 5:56 
GeneralRe: Different package model? Pin
Member 932665624-Jan-23 12:22
Member 932665624-Jan-23 12:22 
QuestionHaven't you boys got a day job to do Pin
Sacha Barber13-Dec-22 22:26
Sacha Barber13-Dec-22 22:26 
AnswerRe: Haven't you boys got a day job to do Pin
Sean Ewington14-Dec-22 4:12
staffSean Ewington14-Dec-22 4:12 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.