Package Delivery Detection using CodeProject.AI Server and Blue Iris

So far in my previous articles, we've been doing a lot of setting things up. In this article, I'd like to take what we've learned so far and use CodeProject.AI Server and Blue Iris for something practical, like detecting a delivery package that triggers an alert to send an email notification. Let's dive in.

• Getting the Package Custom Model
• Setting up Blue Iris Alerts
• Detecting a Package - Part 1
  • An Error Sending Email Notifications
• Detecting a Package - Part 2

Getting the Package Custom Model

If you haven't yet set up Blue Iris and CodeProject.AI Server, please check out my article, How to Setup Blue Iris and CodeProject.AI Server on Windows Using a Wyze Cam.

First, we need to get a model that detects packages. The great Mike Lud, CodeProject Community Engineer is training many models and has developed a model that detects packages. So the first step is to download the package model, and put it in the custom model folder. Go to Mike Lud's GitHub, and download package.pt. Then, copy package.pt into the custom model folder for CodeProject.AI Server, which is C:\Program Files\CodeProject\AI\modules\ObjectDetectionYolo\custom-models.

Setting up Blue Iris Alerts

Now, launch Blue Iris. The next step is to set up the email server. Go to the Blue Iris main settings, then the Emails servers tab. Hit the + symbol to enter the email settings.

Enter a description for the email profile. In my case, I have a throwaway Outlook account I sometimes use, so my Description is "Outlook Stevan". From the Preset dropdown, choose "Windows Live Hotmail." There are multiple options here, including AOL Mail, Gmail, Yahoo, and Zoho. Choosing one of the presets automatically fills in the Server address. You can also choose User defined, but you'll need to know the server address in that case.

Next, input the From email (I use the same email to which the alert is going), and enter the Login email (which is the email you wish to receive the notification), and the password for that email account. Then hit Test.

Perfect! Email notifications are working. Hit OK to exit the email profile, then OK again to exit the Blue Iris settings.

Now it's time to set up the alerts. Go to the camera settings, and the Alerts tab. I actually left all the main settings in the alerts menu default. From here, click On Alert.

Then click the + icon to add a new alert. A dropdown loads. From there, select Send email.

In the Required AI objects field, input "package". Then make sure the Outgoing (SMTP) server has the profile selected. In my case, "Outlook Stevan."

In the Recipient(s) field, input the email address that will receive the alert. Then make the subject and body of the email whatever you wish. Hit OK to exit the email configuration alert settings, OK to exit the Action set menu, and OK to accept the camera settings.

All right, let's detect some packages! My wife doesn't want the camera setup at the door so it's just in my office. But thankfully, I have some packages to test.

From the AI tab in the Blue Iris settings, click Open AI control panel to launch CodeProject.AI Server. Also make sure that the Use custom model folder box is checked, since that's where packages.pt is located.

Next, go to the Trigger tab in the camera settings, then hit Artificial Intelligence.

Make sure To confirm says "package" and Custom models also says "package". Hit OK to accept the Artificial Intelligence settings, then OK to accept the camera settings.

Detecting a Package - Part 1

From CodeProject.AI Server, we don't really need anything other than Object Detection (YOLO), so disable everything else by clicking on the three dots next to the process and picking Stop from the dropdown menu.

The camera is on the floor because I couldn't get it higher without mounting it or putting the camera upside down. I'm trying to simulate a front door, so this should be good enough. Time to place a package in the camera's view!

I took a package and placed it the camera's view. And that's it! Package detected!

Now to check Outlook to see if an email alert was sent.

There's the test email, and the email alert! Awesome!

One final test. Let's huck another package in there, and see if it can detect a second package.

An Error Sending Email Notifications

This is where I experienced some troubles. For awhile, putting various packages in front of the camera did trigger an alert. But it was sporadic. Part of the problem was my package. The first package detected was a box. The box was pretty reliably detected every time. The next package I used was a large envelope. The envelope was detected less frequently. In addition, even if I could see the package was detected in Blue Iris, and confirm an alert was triggered, the email notifications stopped working. So I went back to the Email Servers tab in the main Blue Iris settings and hit the Test button again. I got an error:

Error with sending data - data confirmation; Server says: 554 5.2.0
STOREDRV.Submission.Exception:OutboundSpamException; Failed to process message due to a
permanent exception with message [BeginDiagnosticData]WASCL UserAction verdict is not
None. Actual verdict is RefuseQuota, ShowTierUpgrade. OutboundSpamException: WASCL
UserAction verdict is not None. Actual verdict is RefuseQuota, ShowTierUpgrade.
[EndDiagnosticData] [Hostname=FR1P15201MB2167.LAMP152.PROD.OUTLOOK.COM]

I asked for some help understanding the error on the ipcamtalk forum and apparently email service providers have email sending limits. This article details more specifics, but what likely happened in my case is that because I was using an unverified, throwaway Outlook account that I barely used, the emails were no longer sending because I hit my limit for number of emails sent.

Detecting a Package - Part 2

So I decided to setup a Gmail account and see if that helped. I created a new email profile by hitting the + symbol, and followed the same process as before. The only difference is that Gmail requires Oauth authentication rather than a password.

Click the Sign in with Google button, then follow the instructions.

Now to start over and put a package in front of the camera, and see if Gmail detects it.

Success! Package detected, alert triggered, and Gmail notification sent.

One final test (for real this time). Let's huck another package in the camera view and see if the second package is detected. It's the holiday season and the overworked Amazon employees will take any shortcut they can.

Huck!

Bingo! The second package was detected, an alert was triggered, and a Gmail notification was sent. I believe I have successfully set up CodeProject.AI Server to detect delivery packages using Blue Iris.

If you should have any questions or problems with your setup, please feel free to ask them in the article forum below.