批处理怎样提取txt文档中的特定字符串？

batbat001 · 发表于 2023-2-8 18:21:55

本帖最后由 batbat001 于 2023-2-8 22:43 编辑

{"[{"Title":"敏儿演剧史:剧本类","Summary":"上海 : 商务印书馆, **二十二年十月[1933.10]印行 : 王云五发行","PageSum":"101页","Id":"4b1e7731b001378fe85e3470b23e19ab","Pid":"f721283beed81168a7f56770c2e37f42"},{""Title":"上海新学会社图书目录","Summary":"上海 : 上海新学会社, [1936]","PageSum":"1册","Id":"18046902990da9f8c0e3b7081c62b7d9","Pid":"e24b0fcfdd69228163ee2a40179b3a10"},{""Title":"京师译学馆生理卫生学讲义","Summary":"[出版地不详] : [出版者不详], [19??]","PageSum":"1册","Id":"56373fe68f3834c486e0edfd3682ed96","Pid":"81d7172b2f060a842cc78a0486c16647"},{"Title":"中国中部奥陶纪头足类化石","Summary":"[北京] : 农矿部地质调查所 : 国立北平研究院地学研究, **十九年七月[1930.7]印行","PageSum":"18,101页","Id":"0b6ea856ab6a52c74415ced060d1028c","Pid":"7c902869e5461cc9bdb309aa3a0fd9a7"}

有若干txt文档，格式均为utf-8，每个txt里面的内容大致如上，为一些图书的目录信息，所有目录信息都混放在了同一行。
需求：提取每本书的信息到一个excel表格中，包括"Title"、"Summary"、"PageSum"、"Id"、"Pid“字段的内容信息，每组信息一行。

或者，能否把每个txt混在一行的所有书的拆开，把每本书的信息变为单独的一行。这样我可以手动导入excel。

希望能得到大神出手。谢谢了！

terse · 发表于 2023-2-8 19:29:17

本想用JS提取的看了一下发现这个文档不知道是不是没贴全还是怎么回事，开始三个字符 {"[ 和引号问题 ""Title" 会在获取数据时出错

batbat001 · 发表于 2023-2-8 20:02:07

回复 2# terse

上传附件总是不成功，所以剪切了一部分，应该是没剪全。

我尝试着用grep，可是总是搞不定。:'(

hfxiang · 发表于 2023-2-8 20:17:15

本帖最后由 hfxiang 于 2023-2-8 20:18 编辑

将

BEGIN {
RS = "},{"
print ""Title","Summary","PageSum","Id\tPid""
}
{
Title = gensub(/^.*"Title":"([^"]+)",".*$/, ""\\1"", "g", $0)
Summary = gensub(/^.*"Summary":"([^"]+)",".*$/, ""\\1"", "g", $0)
PageSum = gensub(/^.*"PageSum":"([^"]+)",".*$/, ""\\1"", "g", $0)
Id = gensub(/^.*"Id":"([^"]+)",".*$/, ""\\1"", "g", $0)
Pid = gensub(/^.*"Pid":"([^"]+)".*$/, ""\\1"", "g", $0)
print Title "," Summary "," PageSum "," Id "," Pid
}

复制代码

保存为a.awk
下载gawk（ http://bcn.bathome.net/tool/4.1.0/gawk.exe ）,执行

gawk -f.\a.awk 输入文本.txt>输出文本.txt

复制代码

结果

"Title","Summary","PageSum","Id Pid"
"敏儿演剧史：剧本类","上海：商务印书馆，** 二十二年十月 [1933.10] 印行：王云五发行","101页","4b1e7731b001378fe85e3470b23e19ab","f721283beed81168a7f56770c2e37f42"
"上海新学会社图书目录","上海：上海新学会社，[1936]","1 册","18046902990da9f8c0e3b7081c62b7d9","e24b0fcfdd69228163ee2a40179b3a10"
"京师译学馆生理卫生学讲义","[出版地不详] : [出版者不详], [19??]","1册 ","56373fe68f3834c486e0edfd3682ed96","81d7172b2f060a842cc78a0486c16647"
"中国中部奥陶纪头足类化石","[北京] : 农矿部地质调查所：国立北平研究院地学研究，** 十九年七月 [1930.7] 印行","18,101页","0b6ea856ab6a52c74415ced060d1028c","7c902869e5461cc9bdb309aa3a0fd9a7"

复制代码

batbat001 · 发表于 2023-2-8 20:18:14

{"categoryId":null,"totalCount":380,"pageCount":38,"pageSize":10,"documents":[{"Title":"理学的疗法","Title_F":"理学的疗法","TI":["理学的疗法"],"TitleInitial":"L","CLC":["R454"],"Language":"汉语","Summary":"重庆 : 商务印书馆, **三十四年一月[1945.1]出版兼印行 : 王云五发行","Summary_F":"重庆 : 商务印书馆, **三十四年一月[1945.1]出版兼印行 : 王云五发行","LiteratureCategory":10,"WorkSubject":["物理疗法"],"PageSum":"46页","Dimensions":["18×13cm"],"ProvisionActivityDate":["1945"],"ContributionRole":["著"],"Contribution":["刘雄"],"AU":["刘雄"],"PublicationOrganization":["王云五"],"ProvisionActivityPlace":["重庆"],"ProvisionActivityOrganization":["商务印书馆"],"_version_":1738744139389861888,"showMoreEdition":true,"showEntity":false,"entity":[{"OcrTag":true,"Id":"24175e103f83cd8526d3b1c805eeaf98","Pid":"a7769ea13a60e20e94ce1df28f5bb0c1","LiteratureCategory":10,"_version_":1738744221921181696}],"Abstract":null,"LiteratureCategoryPieceTypeId":101,"viewPermission":true,"hasEntites":true},{"Title":"**哲学三晚明诸儒之学术及其精神","Title_F":"**哲学三晚明诸儒之学术及其精神","TI":["**哲学三晚明诸儒之学术及其精神"],"TitleInitial":"G","CLC":["B248"],"Language":"汉语","Summary":"[出版地不详] : 中央训练团党政高级训练班, [19??]印","Summary_F":"[出版地不详] : 中央训练团党政高级训练班, [19??]印","LiteratureCategory":10,"WorkSubject":["哲学","中国","明代"],"PageSum":"12页","Dimensions":["18×13cm"],"ProvisionActivityDate":["19??"],"ContributionRole":["讲"],"Contribution":["钱穆"],"AU":["钱穆"],"ProvisionActivityPlace":["出版地不详"],"ProvisionActivityOrganization":["中央训练团党政高级训练班"],"_version_":1739945908735311872,"showMoreEdition":false,"showEntity":false,"entity":[{"OcrTag":true,"Id":"31ae82462ecddebba36cca9c355da8db","Pid":"ae257d2a67fd6bf7f1bab6ae092d73f3","LiteratureCategory":10,"_version_":1739945929203515397}],"Abstract":null,"LiteratureCategoryPieceTypeId":101,"viewPermission":true,"hasEntites":true}]}

batbat001 · 发表于 2023-2-8 20:23:37

回复 4# hfxiang

收到。我试一下。感谢感谢！

:)

hfxiang · 发表于 2023-2-8 20:43:41

回复 6# batbat001

如果用gawk4.1（utf-8的支持不稳定）不能正确出结果，可尝试用uawk（对utf-8的支持较好）
下载地址：https://www.aliyundrive.com/s/rHoCCwmAww5 （注意要下载全部文件）
提取密码：1234

batbat001 · 发表于 2023-2-8 20:54:37

回复 7# hfxiang

gawk好像不能正确出结果。我再试一下这个。

Batcher · 发表于 2023-2-8 22:08:58

回复 1# batbat001

如果需要上传文件，可以用阿里云盘或百度网盘。

如果需要上传截图，可以找个图床，例如：
http://bbs.bathome.net/thread-60985-1-1.html

batbat001 · 发表于 2023-2-8 22:40:39

回复 7# hfxiang

非常棒！运行顺畅！非常感谢！

batbat001 · 发表于 2023-2-8 22:41:32

回复 9# Batcher

感谢版主提醒！论坛有你们真好！

WHY · 发表于 2023-2-8 22:57:48

本帖最后由 WHY 于 2023-2-9 10:58 编辑

修改正则表达式模式，取消正则多选分支结构。

<# :
@echo off
PowerShell -C ". ([ScriptBlock]::Create((gc -Literal '%~f0') -Join "`r`n")) '%~dp0'"
pause & exit
#>
param($path);
$reg = '"([TSPI][a-zA-Z]+)":("[^"]*")';
$dic = New-Object 'System.Collections.Generic.Dictionary[string, [Collections.ArrayList]]';
$Hash = @{Title=$true; Summary=$true; PageSum=$true; Id=$true; Pid=$true}
forEach( $file In (dir -Literal $path -Filter *.txt) ){
$str = [IO.File]::ReadAllText($file.FullName, [Text.Encoding]::UTF8);
forEach( $m In [regex]::Matches($str, $reg) ){
$key = $m.Groups[1].Value;
If( $Hash.ContainsKey($key) ){
If( !$dic.ContainsKey($key) ){
$dic[$key] = @();
}
[void]$dic[$key].Add($m.Groups[2].Value);
$count = $dic[$key].Count;
}
}
}
$out = [Collections.ArrayList]@();
[void]$out.Add('"' + ($dic.Keys -join '","') + '"');
for( $i=0; $i -lt $count; $i++ ){
$str = '';
forEach( $key In $dic.Keys ){
$str += $dic[$key][$i] + ',';
}
[void]$out.Add($str);
}
[IO.File]::WriteAllLines('result.csv', $out, [Text.Encoding]::UTF8);

复制代码

terse · 发表于 2023-2-9 04:05:50

@if (0) == (0) echo off
dir /b/a-d *.txt|cscript -nologo -e:jscript "%~0"
pause & exit
@end
function adoText(path) {
var stream,content;
stream = new ActiveXObject("ADODB.Stream");
stream.type = 2;
stream.charset = 'utf-8';
stream.open();
stream.loadFromFile(path);
result = stream.readText(-1);
stream.close();
return result;
}
var fso=new ActiveXObject("Scripting.FileSystemObject");
new_file = 'new_result.csv'
f = fso.CreateTextFile(new_file, true);
f.WriteLine("Title"+','+"Summary"+','+"PageSum"+','+"Id"+','+"Pid");
while (!WSH.StdIn.AtEndOfStream){
var file = WScript.StdIn.Readline();
var content = adoText(file);
var obj = new Function("return" + unescape(content))();
var arr = obj.documents;
for(var i = 0, len = arr.length; i < len; i++ ){
f.WriteLine("""+arr[i].Title+"""+','+"""+arr[i].Summary+"""+','+"""+arr[i].PageSum+"""+','+arr[i].entity[0].Id+','+arr[i].entity[0].Pid);
}
}
f.Close()

复制代码

terse · 发表于 2023-2-10 20:32:10

PS版正则提取

$result = @('"Title","Summary","PageSum","Id","Pid"');
$count = ($result -split(',')).Length
$num = [pscustomobject] @{ Value = 0 }
$reg = '(?<="(Title|Summary|PageSum|(?:I|Pi)d)":)"([^"]*)"';
$group= (select-string *.txt -Pattern $reg -AllMatches -Encoding utf8 ).Matches.Value;
$groups = $($group | Group { [math]::Floor($num.Value++ /$count)})
$result += $groups.ForEach({$_.Group -join(',')})
[IO.File]::WriteAllLines('new_result.csv', $result, [Text.Encoding]::UTF8);

复制代码

WHY · 发表于 2023-2-12 14:48:51

仅处理5楼所列的标准的JSON格式：

@echo off
echo;"Title","Summary","PageSum","Id","Pid">输出.csv
PowerShell "$obj=(gc 输入.txt -Raw -Enc UTF8 | ConvertFrom-Json).documents; for($i=0;$i -lt $obj.Title.Count;$i++){'"'+(@($obj.Title[$i],$obj.Summary[$i],$obj.PageSum[$i],$obj.entity[$i].Id,$obj.entity[$i].Pid) -join '","')+'"'}">>输出.csv
pause

复制代码

		自动登录	找回密码
密码			注册

[新手上路]批处理新手入门导读	[视频教程]批处理基础视频教程	[视频教程]VBS基础视频教程	[批处理精品]批处理版照片整理器
[批处理精品]纯批处理备份&还原驱动	[批处理精品]CMD命令50条不能说的秘密	[在线下载]第三方命令行工具	[在线帮助]VBScript / JScript 在线参考

[文本处理] 批处理怎样提取txt文档中的特定字符串？

评分

评分

评分