2.3人脸识别我也行.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 人脸识别我也行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "硬件准备：摄像头、小音箱\n",
    "\n",
    "注意事项：\n",
    "- 文件夹目录中\"unknown.jpg\"为待测图像，\"face_photo\"文件夹中为来源网络的素材图片,\"xuguai.py\"为本教材封装库。\n",
    "- 请逐步运行单元格中的代码，即可查看具体效果。\n",
    "- 请确认硬件处于上网状态，否则无法使用百度API相关功能。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 准备工作:加载第三方库，填写参数\n",
    "\n",
    "教材专用库提前封装，将百度ai平台的人脸识别接口封装成baidu_face函数，去掉了数据处理的部分\n",
    "\n",
    "这里使用的api账号信息是针对人脸识别应用注册的，如果同学们需要在同一个账号上实现多个功能，可以自行在创建账号时时选择。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import face_recognition as fr\n",
    "import cv2\n",
    "import os\n",
    "import base64\n",
    "from aip import AipFace\n",
    "from xuguai import baidu_face,getpic\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 基础知识:CV2图像处理相关的一些基本函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "#拍照，使用基于opencv库已经封装好的拍照函数\n",
    "getpic(\"test.jpg\")\n",
    "#读取图像，斜杠在Python的字符串中有转义作用，在路径前方加r的作用是保持字符串原始值的意思,这里的图片使用了绝对路径。\n",
    "img = cv2.imread(\"photo.jpg\")  \n",
    "#在图像上添加矩形,这里的四个变量代表矩形四个顶点的位置\n",
    "top,right,bottom,left=(121, 162, 196, 88)\n",
    "cv2.rectangle(img,(left, top),(right, bottom),color = (50,255,50),thickness = 3)\n",
    "#在图像上写字,参数依次为：图像数据，要写入的信息，写入的位置（坐标），使用的字体，字体大小，颜色，字体粗细\n",
    "cv2.putText(img,str('要输入的内容'), (100,100), cv2.FONT_HERSHEY_SIMPLEX , 1.2, (255, 255, 255), 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 体验：趣味年龄识别"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "你今年应该 23 岁了吧\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "-1"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "APP_ID = '15377544'\n",
    "API_KEY = '0qK3r7t8i5SR9ir9wmUsXWMQ'\n",
    "SECRET_KEY = 'hhbbQzWhRF0ejZFeIZ95aGsrZFirlGgP'\n",
    "aipFace = AipFace(APP_ID, API_KEY, SECRET_KEY)  # 填写参数，初始化AipFace对象\n",
    "\n",
    "getpic(\"test.jpg\")\n",
    "#调用人脸识别函数读取图片进行识别\n",
    "result=baidu_face(\"test.jpg\",aipFace)\n",
    "firstman=result['result']['face_list'][0]\n",
    "img = cv2.imread(\"test.jpg\")  \n",
    "cv2.putText(img, str(firstman['age']), (100,100), cv2.FONT_HERSHEY_SIMPLEX , 1.2, (255, 255, 255), 2)\n",
    "cv2.imshow(\"age\",img)\n",
    "cv2.imwrite(\"testout.jpg\",img)\n",
    "print('你今年应该',firstman['age'],'岁了吧')\n",
    "cv2.waitKey(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 应用实例1：百度API人脸识别"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "百度AI平台识别返回结果：\n",
      "{'error_code': 0\n",
      " 'error_msg': 'SUCCESS'\n",
      " 'log_id': 4525897999840\n",
      " 'timestamp': 1586837116\n",
      " 'cached': 0\n",
      " 'result': {'face_num': 1\n",
      " 'face_list': [{'face_token': 'f775a1735f84735d96b612a6365816fd'\n",
      " 'location': {'left': 159.47\n",
      " 'top': 87.6\n",
      " 'width': 94\n",
      " 'height': 89\n",
      " 'rotation': -2}\n",
      " 'face_probability': 1\n",
      " 'angle': {'yaw': 26.71\n",
      " 'pitch': 12.58\n",
      " 'roll': -6.82}\n",
      " 'age': 25\n",
      " 'expression': {'type': 'smile'\n",
      " 'probability': 0.99}\n",
      " 'beauty': 70.25}]}}\n",
      "预计年龄: 25\n"
     ]
    }
   ],
   "source": [
    "from aip import AipFace\n",
    "from xuguai import baidu_face\n",
    "\n",
    "\n",
    "#调用人脸识别函数读取图片进行识别\n",
    "result=baidu_face(\"unknown.jpg\",aipFace)\n",
    "print(\"百度AI平台识别返回结果：\")\n",
    "#观察整体的返回值，face_num人脸数量，我们选取其中第一个人的数据，得出被测试者的评分\n",
    "print(str(result).replace(',','\\n'))\n",
    "firstman=result['result']['face_list'][0]     \n",
    "print('预计年龄:',firstman['age'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 应用实例2：face_recognition库中检测人脸函数face_locations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取图片并识别人脸坐标\n",
    "img = fr.load_image_file(\"face_photo/meeting.jpg\")\n",
    "face_locations = fr.face_locations(img)\n",
    "#调用opencv函数加载原图片\n",
    "img = cv2.imread(\"face_photo/meeting.jpg\")\n",
    "# 遍历每个人脸，并标注\n",
    "faceNum = len(face_locations)\n",
    "for i in range(0, faceNum):\n",
    "    top,right,bottom,left=(face_locations[i][0],face_locations[i][1],face_locations[i][2], face_locations[i][3])\n",
    "    cv2.rectangle(img, (left, top),(right, bottom),color = (50,255,50),thickness = 3)\n",
    "# 显示识别结果\n",
    "cv2.imshow(\"recognition\", img)\n",
    "# waitkey函数为窗口持续时间，单位为毫秒，参数0为永久保持窗口\n",
    "cv2.waitKey(0)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 应用实例3：人脸比较"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "liuxiang False\n",
      "zhangjike False\n",
      "yaoming True\n",
      "The person is:yaoming\n"
     ]
    }
   ],
   "source": [
    "#加载人脸照片\n",
    "liuxiang_image = fr.load_image_file(\"face_photo/subingtian.jpg\");\n",
    "zhangjike_image = fr.load_image_file(\"face_photo/zhangjike.jpg\");\n",
    "yaoming_image = fr.load_image_file(\"face_photo/yaoming.jpg\");\n",
    "unknown_image = fr.load_image_file(\"unknown.jpg\");\n",
    "#将图片数据转化为向量\n",
    "liuxiang_encoding = fr.face_encodings(liuxiang_image)[0]\n",
    "zhangjike_encoding = fr.face_encodings(zhangjike_image)[0]\n",
    "yaoming_encoding = fr.face_encodings(yaoming_image)[0]\n",
    "unknown_encoding = fr.face_encodings(unknown_image)[0]\n",
    "#比较函数face_recognition.compare_faces\n",
    "results = fr.compare_faces([liuxiang_encoding, zhangjike_encoding, yaoming_encoding], unknown_encoding )\n",
    "labels = ['liuxiang', 'zhangjike','yaoming']\n",
    "for i in range(0, len(results)):\n",
    "    print(labels[i]+\" \"+str(results[i]))\n",
    "#输出识别结果为True的照片所对应的标签\n",
    "for i in range(0, len(results)):\n",
    "    if results[i] == True:\n",
    "        print('The person is:'+labels[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 实践活动1：会“数人脸”的机器人\n",
    "\n",
    "除了能听会说，我们还学会了如何让机器人“看到”外面的世界，当我们的机器人“睁开双眼”，看到摄像头前的人脸，能否说出人脸的数量呢？\n",
    "\n",
    "实践内容：识别当前摄像头前的人脸数量，并且通过语音合成功能进行播报。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实践步骤1：实时录像"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "video_capture = cv2.VideoCapture(0)\n",
    "\n",
    "while True:\n",
    "    ret, frame = video_capture.read()\n",
    "    cv2.imshow('Video', frame)\n",
    "    #按q退出\n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break\n",
    "video_capture.release()\n",
    "cv2.destroyAllWindows()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实践步骤2：人脸识别\n",
    "\n",
    "在步骤1的基础上，调用摄像头进行拍摄，对照片进行人脸识别操作，当照片中出现人脸时，播报人脸数量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aip import AipSpeech\n",
    "\"\"\" 语音合成的 APPID AK SK （无法用于人脸识别，如需同时使用这两个功能，尝试在百度平台进行注册）\"\"\"\n",
    "APP_ID = \"19612357\"\n",
    "API_KEY = \"om90mVUHZfck1sviVg1vuGVW\"\n",
    "SECRET_KEY = \"5qy7wvLCRRGGqk22roDkmjBNrfyEcTe1\"\n",
    "\"\"\" 这里是参数设置，请不要改动 \"\"\"\n",
    "wavdict = {'CHUNK':1024,'CHANNELS':1,'RATE':16000,'RECORD_SECONDS':3}\n",
    "client = AipSpeech(APP_ID, API_KEY, SECRET_KEY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import os\n",
    "import time\n",
    "from xuguai import text_to_audio\n",
    "\n",
    "video_capture = cv2.VideoCapture(0)\n",
    "while True:\n",
    "    ret, frame = video_capture.read()\n",
    "    # 读取图片并识别人脸坐标\n",
    "    face_locations = fr.face_locations(frame)\n",
    "    # 遍历每个人脸，并标注\n",
    "    faceNum = len(face_locations)\n",
    "    frame = cv2.putText(frame, 'facenum:'+str(faceNum), (100,100), cv2.FONT_HERSHEY_SIMPLEX,1.2, (255, 255, 255), 2)\n",
    "    cv2.imshow('Video', frame)\n",
    "    #如果屏幕中出现人脸\n",
    "    if faceNum!=0:\n",
    "        text_to_audio('人脸的数量是'+ str(faceNum),'audio.wav',client)\n",
    "        os.system('audio.wav')\n",
    "        # 得到文件大小，单位为KB\n",
    "        voicetime = (os.stat('audio.wav').st_size) \n",
    "        time.sleep(voicetime / 2048)\n",
    "         # 等待播放完毕后关闭播放器进程，这里的默认播放器是wmplayer，同理，还可能是QQPlayer等播放器\n",
    "        os.system('taskkill /F /IM wmplayer.exe') \n",
    "        time.sleep(1)\n",
    "        # 删除文件\n",
    "        os.remove('audio.wav')  \n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break\n",
    "video_capture.release()\n",
    "cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实践步骤3：功能优化\n",
    "\n",
    "* 想要完成播报的效果，不一定要重复调用语音合成功能。比如常见的收款播报提醒，成功支付后播放：“账户收款：321元”。前面固定部分的音频可以预先合成，而数字的合成也只要录制好数字语音和金钱单位。使用时，根据用户输入组合输出即可。\n",
    "* 完善系统，将人脸框出，并在图像中标注数字等信息。\n",
    "\n",
    "（教师参考）\n",
    "for i in range(0, faceNum):\n",
    "    top,right,bottom,left=(face_locations[i][0],face_locations[i][1],face_locations[i][2], face_locations[i][3])\n",
    "    cv2.rectangle(frame, (left, top),(right, bottom),color = (50,255,50),thickness = 3)\n",
    "frame = cv2.putText(frame, 'facenum:'+str(faceNum), (100,100), cv2.FONT_HERSHEY_SIMPLEX,1.2, (255, 255, 255), 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结果呈现：\n",
    "\n",
    "运行代码，尝试多人在摄像头前出现，测试程序是否能够正常运行。\n",
    "\n",
    "思考能否制作出一个完全离线的播报应用，在实际生活中，一个离线的检测器相比联网版本，能够帮我们节省网络带宽、排除因网络状况而产生的各种问题。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 【思考】\n",
    "\n",
    "想要完成播报的效果，不一定要重复调用语音合成功能。比如常见的收款播报提醒，成功支付后播放：“账户收款：321元”。前面固定部分的音频可以预先合成，而数字的合成也只要录制好数字语音和金钱单位。使用时，根据用户输入组合输出即可。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 实践活动2：校园智能门禁系统\n",
    "\n",
    "随着技术的发展，许多学校的校门口都换上了人脸识别系统，用来统计同学们的出入校情况，让家长和老师们能够第一时间知悉。我们也可以动手制作一个，专门识别小组内成员的智能门禁系统。\n",
    "\n",
    "实践内容：制作一个能够批量导入小组成员的人脸信息，认出特定人脸的名称，识别出对应姓名的校园智能门禁系统。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实践步骤1：数据收集，拍照时应保证光线充足，使拍出来的照片清晰可识别\n",
    "\n",
    "用基本的CV2拍照代码，将小组成员照片存入本地文件夹中，注意文件路径是否存在。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#拍照，使用基于opencv库已经封装好的拍照函数\n",
    "getpic(\"team_photo\\name.jpg\")  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实践步骤2：批量读取文件夹内的照片\n",
    "\n",
    "通过比较函数face_recognition.compare_faces，将文件夹内的所有照片与\"unknown.jpg\"进行对比，facecompare函数一次可以最多读取二十张照片进行比较。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "文件夹中的人脸列表 ['subingtian.jpg', 'subingtian2.jpg', 'yaoming.jpg', 'zhangjike.jpg']\n",
      "results:[False, False, True, False]\n",
      "The person is:yaoming.jpg\n"
     ]
    }
   ],
   "source": [
    "import face_recognition as fr\n",
    "\n",
    "num=0\n",
    "teamface=[]\n",
    "filePath = \"face_photo\"\n",
    "\n",
    "#创建一个装有姓名的列表\n",
    "name = os.listdir(filePath)\n",
    "print('文件夹中的人脸列表',name)\n",
    "for i in name:\n",
    "    num=num+1\n",
    "    img=fr.load_image_file(\"face_photo/\" + i )\n",
    "    img=fr.face_encodings(img)[0]\n",
    "    teamface.insert(num,img)\n",
    "    \n",
    "#导入未知图片\n",
    "unknown_image = fr.load_image_file(\"unknown.jpg\");\n",
    "unknown_encoding = fr.face_encodings(unknown_image)[0]\n",
    "\n",
    "#比较函数face_recognition.compare_faces\n",
    "results = fr.compare_faces(teamface,unknown_encoding)\n",
    "labels = name\n",
    "\n",
    "#输出结果\n",
    "print('results:'+str(results))\n",
    "for i in range(0, len(results)):\n",
    "    if results[i] == True:\n",
    "        print('The person is:'+labels[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实践步骤3：功能优化\n",
    "\n",
    "识别出特定人员后，作出反馈，比如指示灯亮起，播报欢迎提示等。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结果呈现：\n",
    "\n",
    "完成小组成员照片导入后，尝试验证效果，看是否会出现误判的情况。如果有这种情况出现，我们应该如何解决?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}