CodeProject.AI Discussions

Issue with the TFLite module and Docker persistent storage

17-May-23 4:48

We've fixed this in a new release due out today.

cheers
Chris Maunder

kyungkyu.112-May-23 17:06

kyungkyu.1

12-May-23 17:06

I was able to get the Docker image running on my Raspberry Pi 4 with TFLite and a Coral USB device. I am running version 2.1.8-Beta. I prefer to manage my containers via Docker Compose. I created my compose file (below) during the first run of the container, everything works perfectly.

However if the container is restarted for any reason, the TFLite module no longer starts correctly. I am seeing the following errors logged:

22:47:27:objectdetection_tflite_adapter.py: Exception ignored in: 
22:47:27:objectdetection_tflite_adapter.py: Traceback (most recent call last):
22:47:27:objectdetection_tflite_adapter.py:   File "/app/modules/ObjectDetectionTFLite/bin/linux/python39/venv/lib/python3.9/site-packages/tflite_runtime/interpreter.py", line 124, in __del__
22:47:27:objectdetection_tflite_adapter.py:     if self._library is not None:
22:47:27:objectdetection_tflite_adapter.py: AttributeError: 'Delegate' object has no attribute '_library'
22:47:27:objectdetection_tflite_adapter.py: Exception ignored in: 
22:47:27:objectdetection_tflite_adapter.py: Traceback (most recent call last):
22:47:27:objectdetection_tflite_adapter.py:   File "/app/modules/ObjectDetectionTFLite/bin/linux/python39/venv/lib/python3.9/site-packages/tflite_runtime/interpreter.py", line 124, in __del__
22:47:27:objectdetection_tflite_adapter.py:     if self._library is not None:
22:47:27:objectdetection_tflite_adapter.py: AttributeError: 'Delegate' object has no attribute '_library'

**  Docker Compose File **

version: '3.9'
services:
  code-project-ai:
    container_name: code-project-ai
    privileged: true
    restart: unless-stopped
    image: codeproject/ai-server:arm64
    devices:
      - /dev/bus/usb:/dev/bus/usb
    volumes:
      - ./ai:/etc/codeproject/ai
      - ./modules:/app/modules
    ports:
      - '32168:32168'

Just a guess, but it seems like some required data is not being persisted? Can anyone please point me in the right direction?

Re: Issue with the TFLite module and Docker persistent storage

Chris Maunder13-May-23 6:39

Re: Issue with the TFLite module and Docker persistent storage

13-May-23 6:39

That all looks good (my previous post said use privileged and don't forget /dev/bus/usb:/dev/bus/us but you already do that).

No idea unfortunately

cheers
Chris Maunder

kyungkyu.117-May-23 2:13

kyungkyu.1

17-May-23 2:13

Any suggestions to pull more logging or test anything else? I already had the logging slider to the debug setting.

Re: Issue with the TFLite module and Docker persistent storage

Chris Maunder23-May-23 8:30

Re: Issue with the TFLite module and Docker persistent storage

23-May-23 8:30

So just to clarify the first report: When you start it, it's all good. When it's restarted it fails?

cheers
Chris Maunder

kyungkyu.123-May-23 11:10

kyungkyu.1

23-May-23 11:10

Yes Chris, that's exactly correct.

2.18 easier to install than 2.16?

Rick Marshall 202312-May-23 17:41

12-May-23 17:41

Hi- I tried to install 2.16 but got stuck so I went back to 2.08. (I was too lazy to follow Chris M's detailed instructions from the sticky.) Is 2.18 more plug and play? Thanks!

Chris Maunder13-May-23 6:35

13-May-23 6:35

You hurt me, man. You hurt me deep. Wink | ;)

Just uninstall 2.0.8, then install 2.1.8 (not 2.1.6). Easy peasy.

cheers
Chris Maunder

Rick Marshall 202322-May-23 14:31

22-May-23 14:31

I tried it again with 2.19-same problem as before. I think my GPU (Quadro K2200) is ok. Frown | :(

17:01:27:Started Object Detection (YOLOv5 6.2) module
17:01:28:detect_adapter.py: Traceback (most recent call last):
17:01:28:detect_adapter.py: File "C:\Program Files\CodeProject\AI\modules\ObjectDetectionYolo\detect_adapter.py", line 20, in
17:01:28:detect_adapter.py: from detect import init_detect, do_detection
17:01:28:detect_adapter.py: File "C:\Program Files\CodeProject\AI\modules\ObjectDetectionYolo\detect.py", line 9, in
17:01:28:detect_adapter.py: from yolov5.models.common import DetectMultiBackend, AutoShape
17:01:28:detect_adapter.py: ModuleNotFoundError: No module named 'yolov5'

Chris Maunder23-May-23 8:28

23-May-23 8:28

Is there an install.log file in the C:\Program Files\CodeProject\AI\modules\ObjectDetectionYolo folder you can share?

cheers
Chris Maunder

Rick Marshall 202324-May-23 10:32

24-May-23 10:32

Installing CodeProject.AI Analysis Module
========================================================================
CodeProject.AI Installer
========================================================================
CUDA Present...True
Allowing GPU Support: Yes
Allowing CUDA Support: Yes
General CodeProject.AI setup
Creating Directories...Done
Installing module ObjectDetectionYolo
Installing python37 in C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37
Checking for python37 download...Present
Creating Virtual Environment...Python 3.7 Already present
Enabling our Virtual Environment...Done
Confirming we have Python 3.7...present
Ensuring Python package manager (pip) is installed...Done
Ensuring Python package manager (pip) is up to date...Done
Choosing Python packages from requirements.windows.cuda.txt
ERROR: Exception:
Traceback (most recent call last):
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\cli\base_command.py", line 169, in exc_logging_wrapper
status = run_func(*args)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\cli\req_command.py", line 248, in wrapper
return func(self, options, args)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\commands\install.py", line 378, in run
reqs, check_supported_wheels=not options.target_dir
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 93, in resolve
collected.requirements, max_rounds=limit_how_complex_resolution_can_be
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 156, in __bool__
return bool(self._sequence)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 155, in __bool__
return any(self)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 143, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 47, in _iter_built
candidate = func()
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 211, in _make_candidate_from_link
version=version,
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 299, in __init__
version=version,
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 156, in __init__
self.dist = self._prepare()
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 304, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 516, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 593, in _prepare_linked_requirement
hashes,
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 170, in unpack_url
hashes=hashes,
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 107, in get_http_url
from_path, content_type = download(link, temp_dir.path)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\network\download.py", line 134, in __call__
resp = _http_get_download(self._session, link)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\network\download.py", line 117, in _http_get_download
resp = session.get(target_url, headers=HEADERS, stream=True)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\requests\sessions.py", line 600, in get
return self.request("GET", url, **kwargs)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_internal\network\session.py", line 517, in request
return super().request(method, url, *args, **kwargs)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\cachecontrol\adapter.py", line 57, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\requests\adapters.py", line 584, in send
return self.build_response(request, resp)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\cachecontrol\adapter.py", line 84, in build_response
request, response
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\cachecontrol\controller.py", line 410, in update_cached_response
cached_response = self.serializer.loads(request, self.cache.get(cache_url))
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\cachecontrol\serialize.py", line 95, in loads
return getattr(self, "_loads_v{}".format(ver))(request, data, body_file)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\cachecontrol\serialize.py", line 186, in _loads_v4
cached = msgpack.loads(data, raw=False)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\msgpack\fallback.py", line 125, in unpackb
ret = unpacker._unpack()
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\msgpack\fallback.py", line 590, in _unpack
ret[key] = self._unpack(EX_CONSTRUCT)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\msgpack\fallback.py", line 590, in _unpack
ret[key] = self._unpack(EX_CONSTRUCT)
File "C:\Program Files\CodeProject\AI\runtimes\bin\windows\python37\venv\lib\site-packages\pip\_vendor\msgpack\fallback.py", line 603, in _unpack
return bytes(obj)
MemoryError
Installing Packages into Virtual Environment...Success
Ensuring Python package manager (pip) is installed...Done
Ensuring Python package manager (pip) is up to date...Done
Choosing Python packages from requirements.txt
Installing Packages into Virtual Environment...Success
Downloading Standard YOLO models...Expanding...Done.
Downloading Custom YOLO models...Expanding...Done.
Module setup complete
Installer exited with code 0

Rick Marshall 202326-May-23 9:38

way to see if server see's object

26-May-23 9:38

Do you want it in another format?

Member 862730912-May-23 3:40

Member 8627309

12-May-23 3:40

i've tried to setup nvr agent with codeproject.al. i'm wondering if there is a log file that will tell me if a request was made to codeproject to detect an object, at a certain time and what was codeproject response. object = true or maybe object = false. sorry, really really newbie.

Re: way to see if server see's object

Mike Lud12-May-23 3:46

Mike Lud

12-May-23 3:46

You can open the Dashboard and select the trace Logging level.

Re: way to see if server see's object

Member 862730912-May-23 3:49

Member 8627309

12-May-23 3:49

YES ! thank you.

After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Vinblastine11-May-23 21:45

Vinblastine

11-May-23 21:45

Hi all. I’m just wondering if this is a known bug or if it is something to look into.

This occurred whilst running 2.14 and now on 2.18 via docker with both the rpi64 and also the arm docker images. I have the usb coral stick installed on an rpi4/ odroid M1. I’m using Blue Iris on bare metal windows 10 machine. After several hours of use with <100ms detection times the detection times rise to >1000ms and no detections reported on Blue Iris. The problem is fixed by having to restart either the docker container or simply by restarting the TF-Lite module from the codeproject.ai server web page. After this detection times return to <100ms (I’m using the small model size and the sub stream for analysis) .

If it’s a known issue I’ll patiently wait but if you think it’s something wrong with my setup let me know!

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Chris Maunder12-May-23 2:20

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

12-May-23 2:20

I've not experienced this, but I'll add this one to our bug list and see if we can replicate / fix the issue.

Off the bat it sounds like a resource (memory?) depletion issue.

cheers
Chris Maunder

Aaron Van Stralen19-May-23 5:03

Aaron Van Stralen

19-May-23 5:03

I am having exactly the same problem. It occurs after about 12 hours of use, detection times will rise to 1000ms from approximately 100ms. I have version 2.19 with docker on a raspberry pi 3. If I restart TF-lite using the "..." on the web interface, that solves the problem for another 12 hours or so before detection times rise again to 1000ms.

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Aaron Van Stralen20-May-23 17:57

Aaron Van Stralen

20-May-23 17:57

I've played around with it a little more leaving the logs at trace and this is what I've found out.

Memory remains pretty constant from the time it is <100ms through the time I notice it increase to >1000ms. So that doesn't suggest it's a memory issue. Specifically, when I view the memory usage through Portainer, the container for Code Proejct AI shows a constant usage of 700 MB

HOWEVER, I notice in the trace logs lots of: "The interpreter is in use. Please try again later."

Does any of this information help? Is there anything else that might help in resolving this issue?

Thank you

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

xyvyx12-May-23 4:39

xyvyx

12-May-23 4:39

This sounds like what happened on my 2GB rpi64 using version 2.1.4. I should have started a new topic, but I posted about this here:
https://www.codeproject.com/Articles/5322557/CodeProject-AI-Server-AI-the-easy-way?msg=5940955#xx5940955xx

I had a another PC nearby that I put a little Quadro P620 in and I'm now running 2.1.8 w/ CUDA. It seems mostly stable after ~ 5 days.

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Shawn Haywood13-May-23 14:56

Shawn Haywood

13-May-23 14:56

Sorry for this question, but can someone remind me where to see the detection time?

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Mike Lud13-May-23 15:18

Mike Lud

13-May-23 15:18

You can see them on the Dashboard Log. Also when using Explorer.

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Shawn Haywood14-May-23 9:40

Shawn Haywood

14-May-23 9:40

Thank you. I seem to also recall seeing the same info in Blue Iris.

Re: After a while detection times rise from <100ms to >1000ms with Coral on arm/rpi64

Chris Maunder16-May-23 7:11