14.4 USDA食品数据库

每种食物都带有若干标识性属性以及两个有关营养成分和分量的列表。这种形式的数据不是很适合分析工作，因此我们需要做一些规整化以使其具有更好用的形式。

从上面列举的那个网址下载并解压数据之后，你可以用任何喜欢的JSON库将其加载到Python中。我用的是Python内置的json模块：

  2. In [155]: db = json.load(open('datasets/usda_food/database.json'))
  4. In [156]: len(db)
  5. Out[156]: 6636

db中的每个条目都是一个含有某种食物全部数据的字典。nutrients字段是一个字典列表，其中的每个字典对应一种营养成分：

  1. In [157]: db[0].keys()
  2. Out[157]: dict_keys(['id', 'description', 'tags', 'manufacturer', 'group', 'porti
  3. ons', 'nutrients'])
  5. In [158]: db[0]['nutrients'][0]
  6. Out[158]: 
  7. {'description': 'Protein',
  8.  'group': 'Composition',
  9.  'units': 'g',
  10.  'value': 25.18}
  12. In [159]: nutrients = pd.DataFrame(db[0]['nutrients'])
  14. In [160]: nutrients[:7]
  15. Out[160]: 
  16.                    description        group units    value
  17. 0                      Protein  Composition     g    25.18
  18. 1            Total lipid (fat)  Composition     g    29.20
  19. 2  Carbohydrate, by difference  Composition     g     3.06
  20. 3                          Ash        Other     g     3.28
  21. 4                       Energy       Energy  kcal   376.00
  22. 5                        Water  Composition     g    39.28
  23. 6                       Energy       Energy    kJ  1573.00

在将字典列表转换为DataFrame时，可以只抽取其中的一部分字段。这里，我们将取出食物的名称、分类、编号以及制造商等信息：

  1. In [161]: info_keys = ['description', 'group', 'id', 'manufacturer']
  3. In [162]: info = pd.DataFrame(db, columns=info_keys)
  5. In [163]: info[:5]
  6. Out[163]: 
  7.                           description                   group    id  \
  8. 0                     Cheese, caraway  Dairy and Egg Products  1008   
  9. 1                     Cheese, cheddar  Dairy and Egg Products  1009
  10. 2                        Cheese, edam  Dairy and Egg Products  1018   
  11. 3                        Cheese, feta  Dairy and Egg Products  1019   
  12. 4  Cheese, mozzarella, part skim milk  Dairy and Egg Products  1028   
  13.   manufacturer  
  14. 0               
  15. 1               
  16. 2               
  17. 3               
  19. In [164]: info.info()
  20. <class 'pandas.core.frame.DataFrame'>
  21. RangeIndex: 6636 entries, 0 to 6635
  22. Data columns (total 4 columns):
  23. description     6636 non-null object
  24. group           6636 non-null object
  25. id              6636 non-null int64
  26. manufacturer    5195 non-null object
  27. dtypes: int64(1), object(3)
  28. memory usage: 207.5+ KB

现在，为了对全部营养数据做一些分析，最简单的办法是将所有食物的营养成分整合到一个大表中。我们分几个步骤来实现该目的。首先，将各食物的营养成分列表转换为一个DataFrame，并添加一个表示编号的列，然后将该DataFrame添加到一个列表中。最后通过concat将这些东西连接起来就可以了：

顺利的话，nutrients的结果是：

  1. In [167]: nutrients
  2. Out[167]: 
  3.                                description        group units    value     id
  4. 0                                  Protein  Composition     g   25.180   1008
  5. 1                        Total lipid (fat)  Composition     g   29.200   1008
  6. 2              Carbohydrate, by difference  Composition     g    3.060   1008
  7. 3                                      Ash        Other     g    3.280   1008
  8. 4                                   Energy       Energy  kcal  376.000   1008
  9. ...                                    ...          ...
  10. ...      ...    ...
  11. 389350                 Vitamin B-12, added     Vitamins   mcg    0.000  43546
  12. 389351                         Cholesterol        Other    mg    0.000  43546
  13. 389352        Fatty acids, total saturated        Other     g    0.072  43546
  14. 389353  Fatty acids, total monounsaturated        Other     g    0.028  43546
  15. 389354  Fatty acids, total polyunsaturated        Other     g    0.041  43546
  16. [389355 rows x 5 columns]

我发现这个DataFrame中无论如何都会有一些重复项，所以直接丢弃就可以了：

  1. In [168]: nutrients.duplicated().sum()  # number of duplicates
  2. Out[168]: 14179
  4. In [169]: nutrients = nutrients.drop_duplicates()

由于两个DataFrame对象中都有”group”和”description”，所以为了明确到底谁是谁，我们需要对它们进行重命名：

  1. In [170]: col_mapping = {'description' : 'food',
  2.    .....:                'group'       : 'fgroup'}
  4. In [171]: info = info.rename(columns=col_mapping, copy=False)
  6. In [172]: info.info()
  7. <class 'pandas.core.frame.DataFrame'>
  8. RangeIndex: 6636 entries, 0 to 6635
  9. Data columns (total 4 columns):
  10. food            6636 non-null object
  11. fgroup          6636 non-null object
  12. id              6636 non-null int64
  13. manufacturer    5195 non-null object
  14. dtypes: int64(1), object(3)
  16. In [173]: col_mapping = {'description' : 'nutrient',
  17.    .....:                'group' : 'nutgroup'}
  18. In [174]: nutrients = nutrients.rename(columns=col_mapping, copy=False)
  20. In [175]: nutrients
  21. Out[175]: 
  22.                                   nutrient     nutgroup units    value     id
  23. 0                                  Protein  Composition     g   25.180   1008
  24. 1                        Total lipid (fat)  Composition     g   29.200   1008
  25. 2              Carbohydrate, by difference  Composition     g    3.060   1008
  26. 3                                      Ash        Other     g    3.280   1008
  27. 4                                   Energy       Energy  kcal  376.000   1008
  28. ...                                    ...          ...   ...      ...    ...
  29. 389350                 Vitamin B-12, added     Vitamins   mcg    0.000  43546
  30. 389351                         Cholesterol        Other    mg    0.000  43546
  31. 389352        Fatty acids, total saturated        Other     g    0.072  43546
  32. 389353  Fatty acids, total monounsaturated        Other     g    0.028  43546
  33. 389354  Fatty acids, total polyunsaturated        Other     g    0.041  43546
  34. [375176 rows x 5 columns]

我们现在可以根据食物分类和营养类型画出一张中位值图（如图14-11所示）：

  1. In [180]: result = ndata.groupby(['nutrient', 'fgroup'])['value'].quantile(0.5)
  3. In [181]: result['Zinc, Zn'].sort_values().plot(kind='barh')

只要稍微动一动脑子，就可以发现各营养成分最为丰富的食物是什么了：

  1. by_nutrient = ndata.groupby(['nutgroup', 'nutrient'])
  3. get_maximum = lambda x: x.loc[x.value.idxmax()]
  4. get_minimum = lambda x: x.loc[x.value.idxmin()]
  6. max_foods = by_nutrient.apply(get_maximum)[['value', 'food']]
  8. # make the food a little smaller
  9. max_foods.food = max_foods.food.str[:50]

由于得到的DataFrame很大，所以不方便在书里面全部打印出来。这里只给出”Amino Acids”营养分组：

  1. In [183]: max_foods.loc['Amino Acids']['food']
  2. Out[183]: 
  3. nutrient
  4. Alanine                          Gelatins, dry powder, unsweetened
  5. Arginine                              Seeds, sesame flour, low-fat
  6. Aspartic acid                                  Soy protein isolate
  7. Cystine               Seeds, cottonseed flour, low fat (glandless)
  8. Glutamic acid                                  Soy protein isolate
  9.                                        ...                        
  10. Serine           Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
  11. Threonine        Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
  12. Tryptophan        Sea lion, Steller, meat with fat (Alaska Native)
  13. Tyrosine         Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
  14. Name: food, Length: 19, dtype: object