Hugging Face Example
Example of using BYOM with a Hugging Face Model
Our BYOM integration can link with many different types of model ranging from classification to object detection to instance segmentation. So long as you meet the minimum requirements, the sky is the limit!
As an example, below we describe how you can connect into the Facebook Resnet 50 Hugging Face model.
Diagram
Below is a diagram showing how a model stage (using BYOM connected to Hugging Face) can be connected into a simple workflow
- The items are initially either uploaded or registered (if using external storage) into the dataset stage
- The items then pass through the model stage for prelabelling
- Once prelabelling is complete, human annotators and reviewers can then make corrections to the model
- These new annotations can be used to retrain or fine tune the model and so the prelabelling should be even better next time!
Steps Needed
- Firstly, the model will need to be get an api_token from Hugging face and app this to the example Python implementation
- You will then need to host the Flask app somewhere which is accessible via HTTP/HTTPS such as a small EC2 instance
- You will then need to register the model with V7 as per the instructions in our main doc
- Setup a workflow with a model stage like the diagram above. Remember to create the desired classes in V7 and map them in your model stage. If they have the exact same name then they will automatically match.
- Activate the model in the V7 models tab
- Send an item through the workflow
- If everything has been implemented correctly then you should get something like the following:
Request and Response Schema
The parsing of the request and response will vary by model but has been done for you in this example.
Example Python Implementation
In order to run this example, you will need an api_token from Hugging Face and a small server on which to run this Flask app. This is the app you would be running in the server part of the diagram.
import base64
import tempfile
import urllib
import io
from PIL import Image
from flask import Flask, request, jsonify
import json
import requests
api_key = "<V7 API KEY>"
#Models Section
def model_query(data, model_id, api_token):
"""
Sends file for inference in HF
"""
headers = {"Authorization": f"Bearer {api_token}"}
API_URL = f"https://api-inference.huggingface.co/models/{model_id}"
response = requests.request("POST", API_URL, headers=headers, data=data)
return json.loads(response.content.decode("utf-8"))
model_id = "facebook/detr-resnet-50"
api_token = "<Hugging Face API Key>"
#Flask App
app = Flask(__name__)
obj_classes_dict = {
"0": "N/A",
"1": "person",
"10": "traffic light",
"11": "fire hydrant",
"12": "street sign",
"13": "stop sign",
"14": "parking meter",
"15": "bench",
"16": "bird",
"17": "cat",
"18": "dog",
"19": "horse",
"2": "bicycle",
"20": "sheep",
"21": "cow",
"22": "elephant",
"23": "bear",
"24": "zebra",
"25": "giraffe",
"26": "hat",
"27": "backpack",
"28": "umbrella",
"29": "shoe",
"3": "car",
"30": "eye glasses",
"31": "handbag",
"32": "tie",
"33": "suitcase",
"34": "frisbee",
"35": "skis",
"36": "snowboard",
"37": "sports ball",
"38": "kite",
"39": "baseball bat",
"4": "motorcycle",
"40": "baseball glove",
"41": "skateboard",
"42": "surfboard",
"43": "tennis racket",
"44": "bottle",
"45": "plate",
"46": "wine glass",
"47": "cup",
"48": "fork",
"49": "knife",
"5": "airplane",
"50": "spoon",
"51": "bowl",
"52": "banana",
"53": "apple",
"54": "sandwich",
"55": "orange",
"56": "broccoli",
"57": "carrot",
"58": "hot dog",
"59": "pizza",
"6": "bus",
"60": "donut",
"61": "cake",
"62": "chair",
"63": "couch",
"64": "potted plant",
"65": "bed",
"66": "mirror",
"67": "dining table",
"68": "window",
"69": "desk",
"7": "train",
"70": "toilet",
"71": "door",
"72": "tv",
"73": "laptop",
"74": "mouse",
"75": "remote",
"76": "keyboard",
"77": "cell phone",
"78": "microwave",
"79": "oven",
"8": "truck",
"80": "toaster",
"81": "sink",
"82": "refrigerator",
"83": "blender",
"84": "book",
"85": "clock",
"86": "vase",
"87": "scissors",
"88": "teddy bear",
"89": "hair drier",
"9": "boat",
"90": "toothbrush"
}
def url_to_binary(url):
"""
Converts presigned URL to binary encoding.
"""
with tempfile.NamedTemporaryFile() as file:
urllib.request.urlretrieve(url, "temp")
img = Image.open("temp").convert("RGB")
output = io.BytesIO()
img.save(output, format='JPEG')
hex_data = output.getvalue()
return hex_data
#V7 Results Extraction
def v7_results(resultos):
"""
Returns facebook/detr-resnet-50 inference results compatible with the V7 Bring Your Own Model (BYOM) format.
Accepts Results from facebook/detr-resnet-50 model
"""
nupdres = resultos
v7results = []
for res in nupdres:
temp_dict = {}
temp_bb = {}
box = res["box"]
temp_bb["h"] = box["ymax"] - box["ymin"]
temp_bb["w"] = box["xmax"] - box["xmin"]
temp_bb["x"] = box["xmin"]
temp_bb["y"] = box["ymin"]
temp_dict["confidence"] = res["score"]
temp_dict["label"] = res["label"]
temp_dict["name"] = res["label"]
temp_dict["bounding_box"] = temp_bb
v7results.append(temp_dict)
return v7results
@app.route("/")
def home():
return "<h1>Home Page</h1>"
@app.route("/api/classes", methods=["GET"])
def classes():
"""
Returns a list of bounding box classes for use in V7 Bring Your Own Model.
"""
returned_classes = list(obj_classes_dict.values())
v7_list = []
for clas in returned_classes:
temp_dict = {}
temp_dict["name"] = clas
temp_dict["type"] = "bounding_box"
v7_list.append(temp_dict)
return jsonify(v7_list)
@app.route("/infer", methods=["POST"])
def infer():
payload = request.json
try:
image = url_to_binary(payload["image"]["url"])
model_results = model_query(image, model_id, api_token)
converted_results = v7_results(model_results)
return jsonify(
status="succeeded",
results=converted_results
)
except:
return jsonify(status="failed", results=[])
#Run Flask App
if __name__ == "__main__":
app.run(debug=True,host='0.0.0.0',port=8080)
Updated about 1 year ago