某宝评论数据采集

@杨某

1501人浏览 · 2026-01-21 14:22:39

@杨某 · 2026-01-21 14:22:39 发布

明确爬取目标

1.明确目标：

在这里插入图片描述

爬取某宝商品评论

目标网站：https://detail.tmall.com/item.htm?
爬取内容：‘昵称’, ‘评论’,‘日期’。

2.在浏览器中打开开发者工具

点击加载全部评论
点击全部评价
搜索部分评论内容，确定数据接口
数据接口：https://h5api.m.tmall.com/h5/mtop.taobao.rate.detaillist.get/6.0/

代码实现

模拟浏览器发送请求

url = "https://h5api.m.taobao.com/h5/mtop.taobao.rate.detaillist.get/6.0/"
headers = {
    'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',
    'referer':'https://item.taobao.com/',
    'cookie':'cna=a2+SIMKPOj8BASQJimB4Udt2; dnk=tb604782088; tracknick=tb604782088; lid=tb604782088; _l_g_=Ug%3D%3D; unb=3037334820; lgc=tb604782088; cookie1=BxBCu%2B8gR2hllfw5x8CYzEAc2Vdl0PXfWEmoS1BknpQ%3D; login=true; wk_cookie2=13dc229b78896fa228d7bb509c9134a3; cookie17=UNDXmqHuiUMQVg%3D%3D; cookie2=1b4308490d4d3504ddf33e4347d2424f; _nk_=tb604782088; cancelledSubSites=empty; sg=806; t=431dcd7f437fa5f2b00b3ffb20faa6f6; sn=; _tb_token_=7beeb3b66b5de; wk_unb=UNDXmqHuiUMQVg%3D%3D; uc1=cookie21=V32FPkk%2FgihF%2FS5nrepr&cookie14=UoYY61rNZrk%2BSA%3D%3D&existShop=false&cookie15=Vq8l%2BKCLz3%2F65A%3D%3D&pas=0&cookie16=W5iHLLyFPlMGbLDwA%2BdvAGZqLg%3D%3D; uc3=id2=UNDXmqHuiUMQVg%3D%3D&vt3=F8dD29IbuYK%2BlwhFVBU%3D&lg2=VFC%2FuZ9ayeYq2g%3D%3D&nk2=F5RDLAFaG54AaY4%3D; uc4=id4=0%40UgcnC95KsuRKMF8zJSRKgUl64gsI&nk4=0%40FY4I70PYYaruIm3uBmo4msj%2FsuaN6w%3D%3D; sgcookie=E100CZxGy40ZN%2B2V7B2cW3%2FD8t8OGDufCHralKAQ%2BOI7Z2wnhTfsj0m9FFsal7WA3lzGrsrrBIwxdGdt8zJz%2Fm1YJmdrSAfw9y2maQVsiRk2BEaL7t4s0EeArZ0hxS8Fz2sy; csg=f12c3e2a; isg=BHV1IL38cFmVd5WD5tF47hGMhPEv8ikEwVpczPeaMew7zpXAv0I51IPPHJJ49kG8; mtop_partitioned_detect=1; _m_h5_tk=0ccb298940b3e76296e5166f5c7c0104_1768970377891; _m_h5_tk_enc=5161e4c95d3672c5c1c1b1f821138aae; xlly_s=1; tfstk=gqdr3p4Ogbhr8WoM7FfEgA3LWTfRn6oszBsC-eYhPgjov4QhT3tcKWTSe2uF0HI5F_v5-W-fcHNBPBJYTFLpN4CIy_LR96msffteyUCp2EwEjQ7cmaLc-8VnhscHRDqmffG6yYbROEm6F6icuw_htajlKi0V5NEltTVunq7hRu23ZHYmuiQaxJbltifcWNfhxHfnuEjA86bkx602oiQnNfShfS7NEqn6wtZxm3BPjUj4_EdVrLwJziPT1I81UGm10WVHga8FDzJQ_vsMBg61jncba68c8nW6UcPyYeY9KtRUq4tMmp-hkKnULGJHPddfhy2kuBSPIQX4JVTMywxhLKna1a5W3ORye2EDrhsyI_LQ70TVQKWOot4UE_965BBMZbrOcOQe0O9rb7-G4wrdonIX9Lr3LTbAuGgquMWDI0SOSeYUpJBJHZSsy4eLpTXAuGgquJedeEQVf43R.',
}
params = {
    'jsv':'2.7.5',
    'appKey':'12574478',
    't':'1768962852942',
    'sign':'fd8ce896bf3f5cf2e332b20c6783b6b7',
    '_bx-login':'new',
    'api':'mtop.taobao.rate.detaillist.get',
    'v':'6.0',
    'isSec':'0',
    'ecode':'1',
    'timeout':'20000',
    'dataType':'jsonp',
    'valueType':'string',
    'type':'jsonp',
    'callback':'mtopjsonp12',
    'data':'{"showTrueCount":false,"auctionNumId":"825277478881","pageNo":1,"pageSize":20,"orderType":"","searchImpr":"-8","expression":"","skuVids":"","rateSrc":"pc_rate_list","rateType":"","foldFlag":"0"}',
}
response = requests.get(url=url,params=params,headers=headers)
html = response.text
pprint(html)

获取数据

text_json = re.findall('mtopjsonp\d+\((.*)',html)[0][:-1]
json_data = json.loads(text_json)
#print(json_data)
rateList = json_data['data']['rateList']
for rate in rateList:
    dit = {
        '昵称': rate['reduceUserNick'],
        '评论':rate['feedback'],
        '日期': rate['feedbackDate']
    }
    csv_writer.writerow(dit)

保存数据

放在模拟浏览器之前，保证代码美观

f = open('data.csv', mode='w', encoding='utf-8-sig', newline='')
csv_writer = csv.DictWriter(f,fieldnames=[
    '昵称',
    '评论',
    '日期',
])
csv_writer.writeheader()

批量抓取

复制数据接口中的一段：https://h5api.m.tmall.com/h5/mtop.taobao.rate.detaillist.get/6.0/，如这里可以复制get/6.0/在开发者工具中进行搜索，并先不断滑动评论，让其不断加载新的内容可以发现，有接口不断加载出来。（接口网址一般是一样的，只是参数不一样）

在这里插入图片描述
打开接口，对比参数；

在这里插入图片描述

#有参数的问号前基本不变
'''
callback=mtopjsonp17    
data= pageNO变化
sign ：加密参数
t:时间戳：内置模块处理                                                                                               t               sign                                                                                                                                                                 callback=mtopjsonp17      data=                   
https://h5api.m.taobao.com/h5/mtop.taobao.rate.detaillist.get/6.0/?jsv=2.7.5&appKey=12574478&t=1768740228672&sign=5dd52f511a09d37c97be9ea6db8055a7&_bx-login=new&api=mtop.taobao.rate.detaillist.get&v=6.0&isSec=0&ecode=1&timeout=20000&dataType=jsonp&valueType=string&type=jsonp&callback=mtopjsonp9&data=%7B%22showTrueCount%22%3Afalse%2C%22auctionNumId%22%3A%22842527698158%22%2C%22pageNo%22%3A1%2C%22pageSize%22%3A20%2C%22orderType%22%3A%22%22%2C%22searchImpr%22%3A%22-8%22%2C%22expression%22%3A%22%22%2C%22skuVids%22%3A%22%22%2C%22rateSrc%22%3A%22pc_rate_list%22%2C%22rateType%22%3A%22%22%2C%22foldFlag%22%3A%220%22%7D
https://h5api.m.taobao.com/h5/mtop.taobao.rate.detaillist.get/6.0/?jsv=2.7.5&appKey=12574478&t=1768745088186&sign=c634cec9646685df8b82536ab2f48891&_bx-login=new&api=mtop.taobao.rate.detaillist.get&v=6.0&isSec=0&ecode=1&timeout=20000&dataType=jsonp&valueType=string&type=jsonp&callback=mtopjsonp17&data=%7B%22showTrueCount%22%3Afalse%2C%22auctionNumId%22%3A%22842527698158%22%2C%22pageNo%22%3A4%2C%22pageSize%22%3A20%2C%22orderType%22%3A%22%22%2C%22searchImpr%22%3A%22-8%22%2C%22expression%22%3A%22%22%2C%22skuVids%22%3A%22%22%2C%22rateSrc%22%3A%22pc_rate_list%22%2C%22rateType%22%3A%22%22%2C%22foldFlag%22%3A%220%22%7D
https://h5api.m.taobao.com/h5/mtop.taobao.rate.detaillist.get/6.0/?jsv=2.7.5&appKey=12574478&t=1768744937976&sign=5069bdd3121c451ac6731c3196967451&_bx-login=new&api=mtop.taobao.rate.detaillist.get&v=6.0&isSec=0&ecode=1&timeout=20000&dataType=jsonp&valueType=string&type=jsonp&callback=mtopjsonp16&data=%7B%22showTrueCount%22%3Afalse%2C%22auctionNumId%22%3A%22842527698158%22%2C%22pageNo%22%3A3%2C%22pageSize%22%3A20%2C%22orderType%22%3A%22%22%2C%22searchImpr%22%3A%22-8%22%2C%22expression%22%3A%22%22%2C%22skuVids%22%3A%22%22%2C%22rateSrc%22%3A%22pc_rate_list%22%2C%22rateType%22%3A%22%22%2C%22foldFlag%22%3A%220%22%7D
https://h5api.m.taobao.com/h5/mtop.taobao.rate.detaillist.get/6.0/?jsv=2.7.5&appKey=12574478&t=1768742684288&sign=f4e57a6c1820b24a247896f5a377f932&_bx-login=new&api=mtop.taobao.rate.detaillist.get&v=6.0&isSec=0&ecode=1&timeout=20000&dataType=jsonp&valueType=string&type=jsonp&callback=mtopjsonp15&data=%7B%22showTrueCount%22%3Afalse%2C%22auctionNumId%22%3A%22842527698158%22%2C%22pageNo%22%3A2%2C%22pageSize%22%3A20%2C%22orderType%22%3A%22%22%2C%22searchImpr%22%3A%22-8%22%2C%22expression%22%3A%22%22%2C%22skuVids%22%3A%22%22%2C%22rateSrc%22%3A%22pc_rate_list%22%2C%22rateType%22%3A%22%22%2C%22foldFlag%22%3A%220%22%7D
'''

-只有这4个参数发生变化
callback=mtopjsonp17 可以写死
data= pageNO 代表页数变化
sign ：加密参数
t:时间戳：内置模块处理

破解加密参数sign

搜索： sign：
在这里插入图片描述
根据搜索的结果和发起程序中显示的文件，可以确认，sign生成于p_home.js文件中，

打开p_home.js
可以看到sign的产生与r.token,c,u,n.data这几个参数有关，使用了s函数，
(r.token + “&” + c + “&” + u + “&” + n.data)
在源代码中打上断点，s函数和 sign: s之前，同时监视r.token,c,u,n.data这四个参数，刷新页面
在这里插入图片描述
可以得到：

u 代表参数appKey
c 代表时间戳
n.data代表参数中的data
r.token 为cookie中_m_h5_tk：0ccb298940b3e76296e5166f5c7c0104_1768970377891
的前半部分0ccb298940b3e76296e5166f5c7c0104
而s函数是哈希加密函数md5
所以可以重写get_sign()方法，

import hashlib


def get_sign(c,page):
    token = "f094d7171d0495a684188404e95ae6d5"
    u = "12574478"#appkey
    n_data = '{"showTrueCount":false,"auctionNumId":"842527698158","pageNo":%d,"pageSize":20,"orderType":"","searchImpr":"-8","expression":"","skuVids":"","rateSrc":"pc_rate_list","rateType":"","foldFlag":"0"}'%page
    string = token + "&" +str(c) + "&" + u + "&" + n_data
    #r.token + "&" + c + "&" + u + "&" + n.data
    sign = hashlib.md5(string.encode('utf-8')).hexdigest()
    print(sign)
    return sign,n_data

c = 1768757449889#时间戳
page = 1
sign,n_data = get_sign(c,page)
print(sign)
print(n_data)

完整代码

import requests

from pprint import pprint
import re
import json
import csv
import hashlib
import time


def get_sign(c,page):
    token = "14e26a8fe2b07e9add51fb27ea815e3a"
    u = "12574478"#appkey
    n_data = '{"showTrueCount":false,"auctionNumId":"842527698158","pageNo":%d,"pageSize":20,"orderType":"","searchImpr":"-8","expression":"","skuVids":"","rateSrc":"pc_rate_list","rateType":"","foldFlag":"0"}'%page
    string = token + "&" +str(c) + "&" + u + "&" + n_data
    #r.token + "&" + c + "&" + u + "&" + n.data
    sign = hashlib.md5(string.encode('utf-8')).hexdigest()
    print(sign)
    return sign,n_data

f = open('data.csv', mode='w', encoding='utf-8-sig', newline='')
csv_writer = csv.DictWriter(f,fieldnames=[
    '昵称',
    '评论',
    '日期',
])
csv_writer.writeheader()
url = "https://h5api.m.taobao.com/h5/mtop.taobao.rate.detaillist.get/6.0/"
headers = {
    'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',
    'referer':'https://item.taobao.com/',
    'cookie':'cna=a2+SIMKPOj8BASQJimB4Udt2; dnk=tb604782088; tracknick=tb604782088; lid=tb604782088; _l_g_=Ug%3D%3D; unb=3037334820; lgc=tb604782088; cookie1=BxBCu%2B8gR2hllfw5x8CYzEAc2Vdl0PXfWEmoS1BknpQ%3D; login=true; wk_cookie2=13dc229b78896fa228d7bb509c9134a3; cookie17=UNDXmqHuiUMQVg%3D%3D; cookie2=1b4308490d4d3504ddf33e4347d2424f; _nk_=tb604782088; cancelledSubSites=empty; sg=806; t=431dcd7f437fa5f2b00b3ffb20faa6f6; sn=; _tb_token_=7beeb3b66b5de; wk_unb=UNDXmqHuiUMQVg%3D%3D; uc1=cookie21=V32FPkk%2FgihF%2FS5nrepr&cookie14=UoYY61rNZrk%2BSA%3D%3D&existShop=false&cookie15=Vq8l%2BKCLz3%2F65A%3D%3D&pas=0&cookie16=W5iHLLyFPlMGbLDwA%2BdvAGZqLg%3D%3D; uc3=id2=UNDXmqHuiUMQVg%3D%3D&vt3=F8dD29IbuYK%2BlwhFVBU%3D&lg2=VFC%2FuZ9ayeYq2g%3D%3D&nk2=F5RDLAFaG54AaY4%3D; uc4=id4=0%40UgcnC95KsuRKMF8zJSRKgUl64gsI&nk4=0%40FY4I70PYYaruIm3uBmo4msj%2FsuaN6w%3D%3D; sgcookie=E100CZxGy40ZN%2B2V7B2cW3%2FD8t8OGDufCHralKAQ%2BOI7Z2wnhTfsj0m9FFsal7WA3lzGrsrrBIwxdGdt8zJz%2Fm1YJmdrSAfw9y2maQVsiRk2BEaL7t4s0EeArZ0hxS8Fz2sy; csg=f12c3e2a; isg=BHV1IL38cFmVd5WD5tF47hGMhPEv8ikEwVpczPeaMew7zpXAv0I51IPPHJJ49kG8; xlly_s=1; mtop_partitioned_detect=1; _m_h5_tk=14e26a8fe2b07e9add51fb27ea815e3a_1768985350947; _m_h5_tk_enc=f3461d3d7980f0880ca821e1fce71873; tfstk=gRSrF8D90uErQXPGQOxU0SCkNQxJ8HP_UMOBK9XHFQAu98pHY_1D-D67yJyeg6dWP3bWKD5XG6iCFM7xYOBdV8KQe3BJvHV_1l1FeTKKA0v_igJDnpBmqBm3lnqhdWc01lZ1ezvJAtV1PHsetpJHxLAkro22edmHxUmkn-JHdb0nq6X03IpKqBYHKtADKpmkxHfhn-JpiLxHr6X03pdDtbnktCi2QU2Y6hwdYoY9zCXkgcWA0L0B_ldmxAsDUNpNEJnnxiJyziXLi-eWPwX9Wg6UYmK5QZxDLMei8B8N-gtcqSqMkejciKf7d2pV89j92UHKv1-PTFjkuvmpqHbJ0KjgdVdf0Q1h4E2sbetcfFxllrFWR3JNtgCrLDfGhOsJkMPZiBQWBnvFvR0eTejr8XpcMIsdzX0erKp21-y2TEbgaCpGK9uKJUQv3Cw8e23prEp21-yqJ2LRHKR_e8C..',
}
#构建翻页
for page in range(1, 11):
    print(f'正在采集第{page}页数据')
    print('*****'*10)
    c = int(time.time() * 1000)
    sign,n_data = get_sign(c=c,page=page)
    params = {
        'jsv':'2.7.5',
        'appKey':'12574478',
        't':f'{c}',
        'sign':sign,
        '_bx-login':'new',
        'api':'mtop.taobao.rate.detaillist.get',
        'v':'6.0',
        'isSec':'0',
        'ecode':'1',
        'timeout':'20000',
        'dataType':'jsonp',
        'valueType':'string',
        'type':'jsonp',
        'callback':'mtopjsonp10',
        'data':n_data,
    }
    response = requests.get(url=url,params=params,headers=headers)
    html = response.text
    #pprint(html)
    text_json = re.findall('mtopjsonp\d+\((.*)',html)[0][:-1]
    json_data = json.loads(text_json)
    #print(json_data)
    rateList = json_data['data']['rateList']
    for rate in rateList:
        dit = {
            '昵称': rate['reduceUserNick'],
            '评论':rate['feedback'],
            '日期': rate['feedbackDate']
        }
        csv_writer.writerow(dit)

脑启社区

脑启社区是一个专注类脑智能领域的开发者社区。欢迎加入社区，共建类脑智能生态。社区为开发者提供了丰富的开源类脑工具软件、类脑算法模型及数据集、类脑知识库、类脑技术培训课程以及类脑应用案例等资源。

更多推荐

面向逻辑思维的程序设计方法——类脑大模型小脑视觉预处理设计

生物小脑视觉对工程设计的启发总结[8-10,13]：分层处理（小脑皮层统一微电路——工程模块化）；预测-校正（前向模型——工程双层预测）；多源融合（前庭+视觉——工程IMU+相机）；学习适应（VOR增益学习——工程在线标定）[8-10,13]。四条启发的工程化即本文的架构骨架[8-10,13]。小脑视觉预处理的总结[13-16]：它是类脑大模型"快慢双视觉"的执行侧视觉通道——双目构建空间结构、前