请选择 进入手机版 | 继续访问电脑版

Hello Mat

 找回密码
 立即注册
查看: 3833|回复: 0

读取海量图像文件和Label标签,并保存数据

[复制链接]

1288

主题

1514

帖子

90

金钱

管理员

Rank: 9Rank: 9Rank: 9

积分
22589
发表于 2017-8-26 22:23:42 | 显示全部楼层 |阅读模式
读取海量图像文件和Label标签,并保存数据:
(1)准备我们的图像数据,Train文件夹图像+Labels、Test文件夹图像+Labels
(2)读取Train文件夹图像+Labels、Test文件夹图像+Labels存储为Tensorflow需要的数据格式
(3)写入.pkl文件,方便下次加载直接使用;
  1. # -*- coding: utf-8 -*-
  2. """
  3. Created on Sat Aug 26 21:52:50 2017

  4. @author: Administrator
  5. """
  6. import pickle
  7. from PIL import Image
  8. import numpy as np
  9. import os

  10. def load_Img(imgDir,imgFoldName):
  11.     imgs = os.listdir(imgDir+imgFoldName)
  12.     imgNum = len(imgs)
  13.     data = np.empty((imgNum,28,28,1),dtype="uint8")
  14.     for i in range (imgNum):
  15.         # img = Image.open(imgDir+imgFoldName+"/"+imgs[i])
  16.         img = Image.open(imgDir+imgFoldName+"/"+ str(i) + '.png' )
  17.         arr = np.asarray(img,dtype="uint8")
  18.         arr = np.reshape(arr, [28,28,1] )
  19.         data[i,:,:,:] = arr
  20.     return data

  21. def dense_to_one_hot(labels_dense, num_classes):
  22.   """Convert class labels from scalars to one-hot vectors."""
  23.   num_labels = labels_dense.shape[0]
  24.   index_offset = np.arange(num_labels) * num_classes
  25.   labels_one_hot = np.zeros((num_labels, num_classes))
  26.   labels_one_hot.flat[index_offset.astype(int) + labels_dense.ravel().astype(int)] = 1
  27.   return labels_one_hot

  28. def load_Label(LabelDir,LabelFoldName, one_hot, num_classes):
  29.     Labs = os.listdir(LabelDir+LabelFoldName)
  30.     File = open(LabelDir+LabelFoldName + Labs[1])
  31.     File.seek(0,0)
  32.     Dstr = File.read();
  33.     data = str.split(Dstr,',')
  34.     Label = np.array(data, dtype=np.float64)
  35.     File.close()
  36.     if one_hot:
  37.         return dense_to_one_hot(Label, num_classes);
  38.     return Label
  39.    
  40. imgDir = "./Test/"
  41. imgFoldName = "test"
  42. test_images = load_Img("./Test/","test")
  43. train_images = load_Img("./Train/","train")
  44. LabelDir =  "D:/Deep_learning_TensorFlow_python/Learning_DayByDay/Mnist/Test/"
  45. LabelFoldName = ""
  46. train_labels = load_Label("./Train/","", True, 10)
  47. test_labels = load_Label("./Test/","", True, 10)

  48. # 保存数据
  49. pickle_data = open('./PKL_mnist/train_images.pkl','wb');
  50. pickle.dump(train_images, pickle_data);
  51. pickle_data.close();

  52. pickle_data = open('./PKL_mnist/test_images.pkl','wb');
  53. pickle.dump(test_images, pickle_data);
  54. pickle_data.close();

  55. pickle_data = open('./PKL_mnist/train_labels.pkl','wb');
  56. pickle.dump(train_labels, pickle_data);
  57. pickle_data.close();

  58. pickle_data = open('./PKL_mnist/test_labels.pkl','wb');
  59. pickle.dump(test_labels, pickle_data);
  60. pickle_data.close();

  61. print('写入PKL文件完成!!!')
复制代码


百度网盘链接:
链接:http://pan.baidu.com/s/1i4BbOlJ
具体链接在halcom.cn论坛,联系人QQ:3283892722
该论坛是一个学习交流平台,我会逐一的和大家分享学习。
欢迎大家录制视频,你可在论坛进行打赏分享。
视频专用播放器:http://halcom.cn/forum.php?mod=viewthread&tid=258&extra=page%3D1











算法QQ  3283892722
群智能算法链接http://halcom.cn/forum.php?mod=forumdisplay&fid=73
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

Python|Opencv|MATLAB|Halcom.cn ( 蜀ICP备16027072号 )

GMT+8, 2024-3-29 23:42 , Processed in 0.218264 second(s), 21 queries .

Powered by Discuz! X3.4

Copyright © 2001-2021, Tencent Cloud.

快速回复 返回顶部 返回列表