10.2. Model Assertions#
Kang, Daniel, et al. "Model assertions for monitoring and improving ml models." Proceedings of Machine Learning and Systems 2 (2020): 481-496.
!git clone https://github.com/ultralytics/yolov5
!pip install -r yolov5/requirements.txt
!apt install -y libboost-all-dev
!pip install -U pip
!pip install "pybind11[global]"
!pip install git+https://github.com/Koukyosyumei/AIJack
import cv2
import torch
from google.colab.patches import cv2_imshow
from aijack.defense.debugging.assertions import (
MultiBoxAssertionError,
assert_multibox,
)
model = torch.hub.load("ultralytics/yolov5", "yolov5s")
img = cv2.imread("yolov5/data/images/bus.jpg")
result = model(img)
/usr/local/lib/python3.10/dist-packages/torch/hub.py:286: UserWarning: You are about to download and run code from an untrusted repository. In a future release, this won't be allowed. To add the repository to your trusted list, change the command to {calling_fn}(..., trust_repo=False) and a command prompt will appear asking for an explicit confirmation of trust, or load(..., trust_repo=True), which will assume that the prompt is to be answered with 'yes'. You can also use load(..., trust_repo='check') which will only prompt for confirmation if the repo is not already trusted. This will eventually be the default behaviour
warnings.warn(
Downloading: "https://github.com/ultralytics/yolov5/zipball/master" to /root/.cache/torch/hub/master.zip
WARNING ⚠️ 'ultralytics.yolo.v8' is deprecated since '8.0.136' and will be removed in '8.1.0'. Please use 'ultralytics.models.yolo' instead.
WARNING ⚠️ 'ultralytics.yolo.utils' is deprecated since '8.0.136' and will be removed in '8.1.0'. Please use 'ultralytics.utils' instead.
Note this warning may be related to loading older models. You can update your model to current structure with:
import torch
ckpt = torch.load("model.pt") # applies to both official and custom models
torch.save(ckpt, "updated-model.pt")
YOLOv5 🚀 2023-7-30 Python-3.10.6 torch-2.0.1+cu118 CPU
Downloading https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt to yolov5s.pt...
100%|██████████| 14.1M/14.1M [00:00<00:00, 56.4MB/s]
Fusing layers...
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
Adding AutoShape...
result.render()
cv2_imshow(result.ims[0])
obj = result.pandas().xyxy[0]
print(obj)
xmin ymin xmax ymax confidence class name
0 219.176910 407.951538 346.656036 873.833130 0.864783 0 person
1 50.105759 395.405853 239.340012 913.149536 0.860630 0 person
2 673.475220 400.604767 810.000000 875.171875 0.849397 0 person
3 14.653461 219.916214 810.000000 810.014282 0.816330 5 bus
4 0.084969 553.378967 64.705879 874.105225 0.644387 0 person
boxes = obj[["xmin", "ymin", "xmax", "ymax"]].values.tolist()
assert_multibox(boxes)
---------------------------------------------------------------------------
MultiBoxAssertionError Traceback (most recent call last)
<ipython-input-7-2a4fe5168b9b> in <cell line: 2>()
1 boxes = obj[["xmin", "ymin", "xmax", "ymax"]].values.tolist()
----> 2 assert_multibox(boxes)
/usr/local/lib/python3.10/dist-packages/aijack/defense/debugging/assertions/assertions.py in assert_multibox(boxes, counter_threshold, eps)
56 counter[i] += 1
57 if counter[i] >= counter_threshold:
---> 58 raise MultiBoxAssertionError(
59 f"Box {i} overlaps more than {counter_threshold} other boxes."
60 )
MultiBoxAssertionError: Box 0 overlaps more than 2 other boxes.