-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsearch.xml
359 lines (173 loc) · 423 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>【S2VT复现】使用S2VT完成视频描述任务的复现记录</title>
<link href="/2025/01/24/%E3%80%90S2VT%E5%A4%8D%E7%8E%B0%E3%80%91%E4%BD%BF%E7%94%A8S2VT%E5%AE%8C%E6%88%90%E8%A7%86%E9%A2%91%E6%8F%8F%E8%BF%B0%E4%BB%BB%E5%8A%A1%E7%9A%84%E5%A4%8D%E7%8E%B0%E8%AE%B0%E5%BD%95/"/>
<url>/2025/01/24/%E3%80%90S2VT%E5%A4%8D%E7%8E%B0%E3%80%91%E4%BD%BF%E7%94%A8S2VT%E5%AE%8C%E6%88%90%E8%A7%86%E9%A2%91%E6%8F%8F%E8%BF%B0%E4%BB%BB%E5%8A%A1%E7%9A%84%E5%A4%8D%E7%8E%B0%E8%AE%B0%E5%BD%95/</url>
<content type="html"><![CDATA[<h1 id="背景介绍"><a href="#背景介绍" class="headerlink" title="背景介绍"></a>背景介绍</h1><p>视频描述任务（Video Captioning Task）是一个跨模态的研究领域，旨在为给定的视频生成自然语言描述。该任务结合了计算机视觉和自然语言处理技术，具有广泛的应用场景，例如视频内容检索、辅助盲人导航、视频摘要生成以及人机交互。</p><h3 id="任务主要内容"><a href="#任务主要内容" class="headerlink" title="任务主要内容"></a>任务主要内容</h3><ol><li><strong>输入</strong>一个视频，通常包含一系列连续的图像帧以及可能的音频信号。</li><li><strong>输出</strong>一段自然语言描述，准确、连贯地表达视频的内容，包括场景、动作、物体、人物及其关系等。</li><li><strong>挑战</strong><ul><li><strong>视频内容理解</strong>：需要识别视频中的物体、场景、动作和复杂事件。</li><li><strong>时序信息建模</strong>：视频是动态数据，需要捕捉事件的时序关系。</li><li><strong>语言生成</strong>：生成的描述需要语法正确且语义合理。</li><li><strong>多模态信息融合</strong>：需要将视觉和听觉信号与语言生成进行高效结合。</li></ul></li></ol><h3 id="技术背景"><a href="#技术背景" class="headerlink" title="技术背景"></a>技术背景</h3><ul><li><strong>计算机视觉的发展</strong><ul><li>随着深度学习的兴起，卷积神经网络（CNN）等技术在图像分类、目标检测和动作识别任务中取得了显著的进展。这些技术为视频描述提供了视觉特征提取的基础。</li><li>视频作为动态序列数据，需要建模时间信息。3D-CNN、LSTM、Transformer等时间序列模型应运而生，为理解视频内容提供了有效方法。</li></ul></li><li><strong>自然语言处理的进步</strong><ul><li>RNN（如LSTM、GRU）以及基于自注意力机制的Transformer（如BERT、GPT）显著提高了语言生成的能力。</li><li>视频描述任务需要将视频的特征转化为自然语言序列，这种跨模态生成任务依赖于NLP中语义建模和句子生成的技术进步。</li></ul></li><li><strong>多模态学习</strong><ul><li>视频描述是典型的多模态任务，需要将视觉、听觉甚至文本信号融合在一起。这促进了多模态深度学习的研究，例如联合嵌入空间、注意力机制和多模态预训练模型的开发。</li></ul></li></ul><h3 id="应用背景"><a href="#应用背景" class="headerlink" title="应用背景"></a>应用背景</h3><ul><li><strong>信息爆炸与需求增长</strong><ul><li>随着社交媒体和流媒体平台（如YouTube、TikTok）的视频内容急剧增长，手动标注或分类视频内容的效率低下。自动化的视频描述技术可以帮助快速生成视频元数据，便于检索、推荐和管理。</li></ul></li><li><strong>无障碍技术</strong><ul><li>对于视觉障碍者，视频描述可以通过语音生成技术，为他们提供更好的信息获取方式。例如，将视频中的重要事件、动作和场景生成语音描述，提升用户体验。</li></ul></li><li><strong>安全与监控</strong><ul><li>视频监控设备生成的大量数据难以被人工分析。视频描述技术可以对视频内容进行自动总结，用于事件检测、异常分析等。</li></ul></li></ul><h1 id="任务介绍"><a href="#任务介绍" class="headerlink" title="任务介绍"></a>任务介绍</h1><p>在本次实验中，我们主要使用的视频的图像信息，利用视觉系统来抽取特征，并进行描述。我们使用训练视频，训练了一个S2VT模型，并用它为测试集的每条视频生成一个描述语句。</p><p>训练集中，每个视频提供了5个标签语句。但是观察到，5个语句如果不尽相同，那么其实也就说明用一句话来表述视频其实是有所受限的，如何评估一句话是否准确地描述了视频，其实也是一个开放性问题。</p><p>具体实验内容可参考<a href="https://github.com/Warma10032/S2VT">https://github.com/Warma10032/S2VT</a></p><h1 id="S2VT方法介绍"><a href="#S2VT方法介绍" class="headerlink" title="S2VT方法介绍"></a>S2VT方法介绍</h1><p>S2VT 模型（Sequence to Sequence Video to Text）由 Venugopalan 等人在 2015 年首次提出，标志着视频描述生成领域的重要里程碑。这一模型首次将通用的序列到序列（Seq2Seq）框架引入视频描述任务，实现了从视频到文本的端到端生成。</p><h3 id="模型核心思想"><a href="#模型核心思想" class="headerlink" title="模型核心思想"></a>模型核心思想</h3><p>Seq2Seq 模型是一种深度学习架构，最初用于解决序列到序列的映射问题，例如机器翻译、对话生成等任务。在 S2VT 模型中，这一框架被巧妙地应用于视频描述领域：</p><ol><li><strong>编码阶段</strong>：通过一个 LSTM 网络对输入视频帧序列进行编码，将视频帧序列转化为一个固定长度的高维向量表示。</li><li><strong>解码阶段</strong>：另一个 LSTM 网络基于编码后的向量表示，生成与视频内容相对应的自然语言描述。</li></ol><p>这一流程利用了 LSTM 的强大时序建模能力，使得模型能够处理动态、多帧的视频数据并生成符合语法规范的文本。</p><h3 id="输入数据形式"><a href="#输入数据形式" class="headerlink" title="输入数据形式"></a>输入数据形式</h3><ol><li><strong>RGB 图像输入</strong>：<ul><li>通过卷积神经网络（CNN）对每一帧图像提取特征，得到每个帧的特征表示。</li><li>这些特征表示再被输入到 LSTM 网络中进行时序建模。</li><li>RGB 图像保留了丰富的空间信息（例如颜色、纹理和物体形状），是视频描述生成的重要信息源。</li></ul></li><li><strong>光学流输入</strong>：<ul><li>光学流图像表示帧与帧之间的运动信息（例如物体移动和行为特征），是捕捉视频时序信息的关键。</li><li>相较于RGB图像，光学流专注于动态变化，可以直接输入到 LSTM 网络中，以进一步增强时序信息的建模。</li></ul></li></ol><p>通过结合 RGB 和光学流输入，S2VT 模型可以充分利用静态场景信息和动态运动信息，从而提升描述生成的准确性和流畅性。</p><h3 id="模型优势"><a href="#模型优势" class="headerlink" title="模型优势"></a>模型优势</h3><ol><li><strong>处理可变长度的输入帧</strong>：<ul><li>视频帧数量通常是可变的，而传统的 RNN 模型在处理变长序列时容易出现梯度消失或梯度爆炸问题。</li><li>LSTM 网络通过引入门控机制（如遗忘门和输入门），有效地缓解了这些问题，使得模型能够稳定地处理长时间序列。</li></ul></li><li><strong>学习视频的时序结构</strong>：<ul><li>视频是典型的时序数据，帧与帧之间存在强相关性。S2VT 模型通过 LSTM 捕捉这种时序依赖关系，生成的文本描述更加准确且连贯。</li><li>特别是在复杂场景中，模型可以通过分析帧序列的变化动态描述事件的发展。</li></ul></li></ol><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124151603231.jpeg" alt=""></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124151611510.jpeg" alt="S2VT结构图"></p><h1 id="实验内容"><a href="#实验内容" class="headerlink" title="实验内容"></a>实验内容</h1><h3 id="VGG抽取特征器"><a href="#VGG抽取特征器" class="headerlink" title="VGG抽取特征器"></a>VGG抽取特征器</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 读取并处理输入</span></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">VGG16FeatureExtractor</span>(nn.Module):</span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">__init__</span>(<span class="params">self, model</span>):</span><br><span class="line"> <span class="built_in">super</span>(VGG16FeatureExtractor, <span class="variable language_">self</span>).__init__()</span><br><span class="line"> <span class="variable language_">self</span>.features = model.features <span class="comment"># VGG16卷积部分</span></span><br><span class="line"> <span class="variable language_">self</span>.classifier = model.classifier[:<span class="number">6</span>] <span class="comment"># 保留前6层全连接层</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">forward</span>(<span class="params">self, x</span>):</span><br><span class="line"> <span class="comment"># 获取卷积层输出</span></span><br><span class="line"> x = <span class="variable language_">self</span>.features(x)</span><br><span class="line"> <span class="comment"># 展平卷积层的输出为(batch_size, 25088)</span></span><br><span class="line"> x = x.view(x.size(<span class="number">0</span>), -<span class="number">1</span>) <span class="comment"># batch_size, 512 * 7 * 7 -> batch_size, 25088</span></span><br><span class="line"> <span class="comment"># 将展平后的特征传递给全连接层</span></span><br><span class="line"> x = <span class="variable language_">self</span>.classifier(x)</span><br><span class="line"> <span class="keyword">return</span> x</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">extract_feats_for_video</span>(<span class="params">file, batch_size, device, save_dir</span>):</span><br><span class="line"> <span class="string">"""Extract features for a single video."""</span></span><br><span class="line"> <span class="comment"># 加载预训练的VGG-16模型</span></span><br><span class="line"> model = VGG16FeatureExtractor(models.vgg16(pretrained=<span class="literal">True</span>))</span><br><span class="line"> model.<span class="built_in">eval</span>() <span class="comment"># 设置为评估模式</span></span><br><span class="line"> model.to(device) <span class="comment"># 移动模型到GPU或CPU</span></span><br><span class="line"></span><br><span class="line"> preprocess = transforms.Compose([</span><br><span class="line"> transforms.ToTensor(),</span><br><span class="line"> transforms.Normalize(mean=[<span class="number">0.485</span>, <span class="number">0.456</span>, <span class="number">0.406</span>], std=[<span class="number">0.229</span>, <span class="number">0.224</span>, <span class="number">0.225</span>]),</span><br><span class="line"> transforms.Resize((<span class="number">224</span>, <span class="number">224</span>)),</span><br><span class="line"> ])</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 读取视频</span></span><br><span class="line"> vid = imageio.get_reader(<span class="string">f"E:/tzy/zy/深度学习/视频描述/dataset/video/<span class="subst">{file}</span>"</span>, <span class="string">'ffmpeg'</span>)</span><br><span class="line"> curr_frames = []</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> frame <span class="keyword">in</span> vid:</span><br><span class="line"> <span class="comment"># 调整帧大小</span></span><br><span class="line"> frame = skimage.transform.resize(frame, [<span class="number">224</span>, <span class="number">224</span>])</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(frame.shape) < <span class="number">3</span>:</span><br><span class="line"> frame = np.repeat(frame, <span class="number">3</span>).reshape([<span class="number">224</span>, <span class="number">224</span>, <span class="number">3</span>])</span><br><span class="line"> frame = Image.fromarray((frame * <span class="number">255</span>).astype(np.uint8))</span><br><span class="line"> curr_frames.append(preprocess(frame))</span><br><span class="line"></span><br><span class="line"> curr_frames = torch.stack(curr_frames).to(device) <span class="comment"># 将图像移动到 GPU/CPU</span></span><br><span class="line"></span><br><span class="line"> idx = np.linspace(<span class="number">0</span>, <span class="built_in">len</span>(curr_frames) - <span class="number">1</span>, <span class="number">80</span>).astype(<span class="built_in">int</span>) <span class="comment"># 获取80帧</span></span><br><span class="line"> curr_frames = curr_frames[idx]</span><br><span class="line"></span><br><span class="line"> curr_feats = []</span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">0</span>, <span class="number">80</span>, batch_size):</span><br><span class="line"> curr_batch = curr_frames[i:i + batch_size]</span><br><span class="line"> <span class="keyword">with</span> torch.no_grad():</span><br><span class="line"> features = model(curr_batch)</span><br><span class="line"> curr_feats.append(features.cpu().numpy())</span><br><span class="line"></span><br><span class="line"> curr_feats = np.concatenate(curr_feats, axis=<span class="number">0</span>)</span><br><span class="line"> save_path = os.path.join(save_dir, <span class="string">f"<span class="subst">{file[:-<span class="number">4</span>]}</span>.npy"</span>)</span><br><span class="line"> np.save(save_path, curr_feats)</span><br><span class="line"> <span class="built_in">print</span>(<span class="string">f"Saved features for <span class="subst">{file}</span> to <span class="subst">{save_path}</span>"</span>)</span><br><span class="line"></span><br></pre></td></tr></table></figure><ol><li><strong>特征提取器的设计</strong>：<ul><li>基于预训练的VGG16网络构建了一个特征提取器</li><li>保留了VGG16的所有卷积层和前6层全连接层</li><li>去掉了最后一层全连接层，这样输出的是视频的特征表示，而不是分类结果</li></ul></li><li><strong>视频处理流程</strong>：<ul><li>首先读取指定路径下的视频文件</li><li>对视频进行采样，从整个视频中均匀抽取80帧画面</li><li>每一帧都会经过预处理：<ul><li>调整尺寸为224x224（VGG16的标准输入尺寸）</li><li>转换为张量格式</li><li>进行标准化处理（使用ImageNet数据集的均值和标准差）</li></ul></li></ul></li><li><strong>批处理特征提取</strong>：<ul><li>将采样得到的80帧分成多个批次进行处理</li><li>每个批次的帧会同时送入VGG16模型</li><li>使用torch.no_grad()确保不计算梯度，提高处理效率</li><li>模型输出的是每一帧的4096维特征向量（来自倒数第二个全连接层）</li></ul></li><li><strong>特征保存</strong>：<ul><li>将所有批次的特征向量合并</li><li>保存为.npy文件，文件名与视频名对应</li><li>每个视频最终得到一个shape为(80, 4096)的特征矩阵，表示80帧画面的特征</li></ul></li></ol><h3 id="S2VT模型"><a href="#S2VT模型" class="headerlink" title="S2VT模型"></a>S2VT模型</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">S2VT</span>(nn.Module):</span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">__init__</span>(<span class="params">self, vocab_size, batch_size=<span class="number">10</span>, frame_dim=<span class="number">4096</span>, hidden=<span class="number">500</span>, dropout=<span class="number">0.5</span>, n_step=<span class="number">80</span></span>):</span><br><span class="line"> <span class="built_in">super</span>(S2VT, <span class="variable language_">self</span>).__init__()</span><br><span class="line"> <span class="variable language_">self</span>.batch_size = batch_size</span><br><span class="line"> <span class="variable language_">self</span>.frame_dim = frame_dim <span class="comment"># 视频特征维度</span></span><br><span class="line"> <span class="variable language_">self</span>.hidden = hidden <span class="comment"># 隐藏层维度</span></span><br><span class="line"> <span class="variable language_">self</span>.n_step = n_step <span class="comment"># 时间步长（视频帧数）</span></span><br><span class="line"></span><br><span class="line"> <span class="variable language_">self</span>.drop = nn.Dropout(p=dropout)</span><br><span class="line"> <span class="variable language_">self</span>.linear1 = nn.Linear(frame_dim, hidden) <span class="comment"># 视频特征降维</span></span><br><span class="line"> <span class="variable language_">self</span>.linear2 = nn.Linear(hidden, vocab_size) <span class="comment"># 输出层，映射到词表大小</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 编码器LSTM</span></span><br><span class="line"> <span class="variable language_">self</span>.lstm1 = nn.LSTM(hidden, hidden, batch_first=<span class="literal">True</span>, dropout=dropout)</span><br><span class="line"> <span class="comment"># 解码器LSTM，输入维度是2*hidden因为包含了视频特征和词嵌入的拼接</span></span><br><span class="line"> <span class="variable language_">self</span>.lstm2 = nn.LSTM(<span class="number">2</span>*hidden, hidden, batch_first=<span class="literal">True</span>, dropout=dropout)</span><br><span class="line"></span><br><span class="line"> <span class="variable language_">self</span>.embedding = nn.Embedding(vocab_size, hidden) <span class="comment"># 词嵌入层</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">forward</span>(<span class="params">self, video, caption=<span class="literal">None</span></span>):</span><br><span class="line"> <span class="comment"># 重塑视频特征维度</span></span><br><span class="line"> video = video.contiguous().view(-<span class="number">1</span>, <span class="variable language_">self</span>.frame_dim)</span><br><span class="line"> video = <span class="variable language_">self</span>.drop(video)</span><br><span class="line"> video = <span class="variable language_">self</span>.linear1(video) <span class="comment"># 视频特征降维</span></span><br><span class="line"> video = video.view(-<span class="number">1</span>, <span class="variable language_">self</span>.n_step, <span class="variable language_">self</span>.hidden)</span><br><span class="line"> <span class="comment"># 在时间维度上填充视频特征</span></span><br><span class="line"> padding = torch.zeros([<span class="variable language_">self</span>.batch_size, <span class="variable language_">self</span>.n_step-<span class="number">1</span>, <span class="variable language_">self</span>.hidden]).cuda()</span><br><span class="line"> video = torch.cat((video, padding), <span class="number">1</span>) <span class="comment"># 视频输入</span></span><br><span class="line"> vid_out, state_vid = <span class="variable language_">self</span>.lstm1(video) <span class="comment"># 通过编码器LSTM</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> <span class="variable language_">self</span>.training:</span><br><span class="line"> <span class="comment"># 训练模式</span></span><br><span class="line"> caption = <span class="variable language_">self</span>.embedding(caption[:, <span class="number">0</span>:<span class="variable language_">self</span>.n_step-<span class="number">1</span>]) <span class="comment"># 对输入的描述文本进行词嵌入</span></span><br><span class="line"> padding = torch.zeros([<span class="variable language_">self</span>.batch_size, <span class="variable language_">self</span>.n_step, <span class="variable language_">self</span>.hidden]).cuda()</span><br><span class="line"> caption = torch.cat((padding, caption), <span class="number">1</span>) <span class="comment"># 描述文本填充</span></span><br><span class="line"> caption = torch.cat((caption, vid_out), <span class="number">2</span>) <span class="comment"># 将视频特征和描述文本拼接</span></span><br><span class="line"></span><br><span class="line"> cap_out, state_cap = <span class="variable language_">self</span>.lstm2(caption) <span class="comment"># 通过解码器LSTM</span></span><br><span class="line"> <span class="comment"># cap_out的大小是 [batch_size, 2*n_step-1, hidden]</span></span><br><span class="line"> cap_out = cap_out[:, <span class="variable language_">self</span>.n_step:, :] <span class="comment"># 只保留生成的描述部分</span></span><br><span class="line"> cap_out = cap_out.contiguous().view(-<span class="number">1</span>, <span class="variable language_">self</span>.hidden)</span><br><span class="line"> cap_out = <span class="variable language_">self</span>.drop(cap_out)</span><br><span class="line"> cap_out = <span class="variable language_">self</span>.linear2(cap_out) <span class="comment"># 映射到词表大小</span></span><br><span class="line"> <span class="keyword">return</span> cap_out</span><br><span class="line"> <span class="comment"># 输出大小 [batch_size*79, vocab_size]</span></span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> <span class="comment"># 测试模式（生成描述）</span></span><br><span class="line"> padding = torch.zeros([<span class="variable language_">self</span>.batch_size, <span class="variable language_">self</span>.n_step, <span class="variable language_">self</span>.hidden]).cuda()</span><br><span class="line"> cap_input = torch.cat((padding, vid_out[:, <span class="number">0</span>:<span class="variable language_">self</span>.n_step, :]), <span class="number">2</span>)</span><br><span class="line"> cap_out, state_cap = <span class="variable language_">self</span>.lstm2(cap_input)</span><br><span class="line"> <span class="comment"># 第二层LSTM的填充输入，80个时间步</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 生成开始符号<BOS>的批处理张量</span></span><br><span class="line"> bos_id = word2id[<span class="string">'<BOS>'</span>]*torch.ones(<span class="variable language_">self</span>.batch_size, dtype=torch.long)</span><br><span class="line"> bos_id = bos_id.cuda()</span><br><span class="line"> cap_input = <span class="variable language_">self</span>.embedding(bos_id)</span><br><span class="line"> cap_input = torch.cat((cap_input, vid_out[:, <span class="variable language_">self</span>.n_step, :]), <span class="number">1</span>)</span><br><span class="line"> cap_input = cap_input.view(<span class="variable language_">self</span>.batch_size, <span class="number">1</span>, <span class="number">2</span>*<span class="variable language_">self</span>.hidden)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 生成第一个词</span></span><br><span class="line"> cap_out, state_cap = <span class="variable language_">self</span>.lstm2(cap_input, state_cap)</span><br><span class="line"> cap_out = cap_out.contiguous().view(-<span class="number">1</span>, <span class="variable language_">self</span>.hidden)</span><br><span class="line"> cap_out = <span class="variable language_">self</span>.drop(cap_out)</span><br><span class="line"> cap_out = <span class="variable language_">self</span>.linear2(cap_out)</span><br><span class="line"> cap_out = torch.argmax(cap_out, <span class="number">1</span>)</span><br><span class="line"> <span class="comment"># 输入"<BOS>"开始生成过程</span></span><br><span class="line"></span><br><span class="line"> caption = []</span><br><span class="line"> caption.append(cap_out)</span><br><span class="line"> <span class="comment"># 将生成的词索引添加到caption列表中，每个时间步为每个批次生成一个词</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 循环生成剩余的词</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="variable language_">self</span>.n_step-<span class="number">2</span>):</span><br><span class="line"> cap_input = <span class="variable language_">self</span>.embedding(cap_out)</span><br><span class="line"> cap_input = torch.cat((cap_input, vid_out[:, <span class="variable language_">self</span>.n_step+<span class="number">1</span>+i, :]), <span class="number">1</span>)</span><br><span class="line"> cap_input = cap_input.view(<span class="variable language_">self</span>.batch_size, <span class="number">1</span>, <span class="number">2</span> * <span class="variable language_">self</span>.hidden)</span><br><span class="line"></span><br><span class="line"> cap_out, state_cap = <span class="variable language_">self</span>.lstm2(cap_input, state_cap)</span><br><span class="line"> cap_out = cap_out.contiguous().view(-<span class="number">1</span>, <span class="variable language_">self</span>.hidden)</span><br><span class="line"> cap_out = <span class="variable language_">self</span>.drop(cap_out)</span><br><span class="line"> cap_out = <span class="variable language_">self</span>.linear2(cap_out)</span><br><span class="line"> cap_out = torch.argmax(cap_out, <span class="number">1</span>) <span class="comment"># 获取词表中概率最高的词的索引</span></span><br><span class="line"> caption.append(cap_out)</span><br><span class="line"> <span class="keyword">return</span> caption <span class="comment"># caption的大小为 [79, batch_size]</span></span><br></pre></td></tr></table></figure><p>这个S2VT模型的工作原理可以分为以下几个关键部分：</p><ol><li><strong>模型结构设计</strong>：<ul><li>包含两个LSTM层：编码器LSTM和解码器LSTM</li><li>使用词嵌入层将词转换为向量表示</li><li>包含视频特征降维层和最终输出层</li><li>使用dropout来防止过拟合</li></ul></li><li><strong>视频特征处理</strong>：<ul><li>首先对输入的视频特征进行降维处理（从4096维降到hidden维）</li><li>通过编码器LSTM处理视频序列</li><li>在时间维度上进行填充，以匹配描述生成的长度需求</li></ul></li><li><strong>训练模式下的工作流程</strong>：<ul><li>将输入的描述文本转换为词嵌入表示</li><li>将视频特征和描述文本进行拼接</li><li>通过解码器LSTM生成预测</li><li>输出每个时间步的词表概率分布</li><li>可以直接用于计算损失函数进行训练</li></ul></li><li><strong>测试模式下的工作流程</strong>（生成描述）：<ul><li>首先输入开始符号<BOS></li><li>逐词生成描述：<ul><li>将上一个生成的词进行词嵌入</li><li>与对应时间步的视频特征拼接</li><li>通过解码器LSTM预测下一个词</li><li>选择概率最高的词作为输出</li></ul></li><li>重复这个过程直到生成完整描述</li></ul></li><li><strong>特点和创新</strong>：<ul><li>采用了编码器-解码器架构</li><li>在解码阶段同时利用了视频特征和文本特征</li><li>可以处理变长的输入视频序列</li><li>能够生成流畅的描述文本</li></ul></li></ol><p>这个模型实现了一个端到端的视频描述生成系统，将视频理解和自然语言生成有机地结合在一起。在训练时使用教师强制（teacher forcing）策略，而在测试时采用自回归方式生成描述。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124151628052.png" alt="训练损失"></p><h3 id="测试集结果示例"><a href="#测试集结果示例" class="headerlink" title="测试集结果示例"></a>测试集结果示例</h3><div class="table-container"><table><thead><tr><th>video id</th><th>row id</th><th>caption</th></tr></thead><tbody><tr><td>G_23245</td><td>2</td><td>people digging soil in the river</td></tr><tr><td>G_23250</td><td>8</td><td>rescue workers and a person in the workshop</td></tr><tr><td>G_23354</td><td>22</td><td>in suits and two women are talking</td></tr><tr><td>G_23514</td><td>38</td><td>many cars driving on the road</td></tr></tbody></table></div>]]></content>
<categories>
<category> 科研日记 </category>
</categories>
<tags>
<tag> 人工智能 </tag>
<tag> 复现 </tag>
<tag> 计算机视觉 </tag>
<tag> 视频描述 </tag>
</tags>
</entry>
<entry>
<title>【SEU-OS-Lab3】东南大学操作系统专题实践三教程</title>
<link href="/2025/01/24/%E3%80%90SEU-OS-Lab3%E3%80%91%E4%B8%9C%E5%8D%97%E5%A4%A7%E5%AD%A6%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E4%B8%93%E9%A2%98%E5%AE%9E%E8%B7%B5%E4%B8%89%E6%95%99%E7%A8%8B/"/>
<url>/2025/01/24/%E3%80%90SEU-OS-Lab3%E3%80%91%E4%B8%9C%E5%8D%97%E5%A4%A7%E5%AD%A6%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E4%B8%93%E9%A2%98%E5%AE%9E%E8%B7%B5%E4%B8%89%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<h2 id="实验要求"><a href="#实验要求" class="headerlink" title="实验要求"></a>实验要求</h2><p>实现具有管道、重定向功能的shell，能够执行一些简单的基本命令，如进程执行、列目录等</p><p>具体要求：</p><ol><li>设计一个C语言程序，完成最基本的shell角色：给出命令行提示符、能够逐次接受命令；<ul><li>对于命令分成三种内部命令（例如help命令、exit命令等）</li><li>外部命令（常见的ls、cp等，以及其他磁盘上的可执行程序HelloWrold等）</li><li>无效命令（不是上述二种命令）</li></ul></li><li>具有支持管道的功能，即在shell中输入诸如“dir | more”能够执行dir命令并将其输出通过管道将其输入传送给more。</li><li>具有支持重定向的功能，即在shell中输入诸如“dir > direct.txt”能够执行dir命令并将结果输出到direct.txt</li></ol><h2 id="实验目的"><a href="#实验目的" class="headerlink" title="实验目的"></a>实验目的</h2><ul><li>通过实验了解Shell实现机制。</li></ul><h2 id="实验环境"><a href="#实验环境" class="headerlink" title="实验环境"></a>实验环境</h2><ul><li>WSL2</li><li>Ubuntu22.04</li></ul><h2 id="实验内容"><a href="#实验内容" class="headerlink" title="实验内容"></a>实验内容</h2><h3 id="基础知识"><a href="#基础知识" class="headerlink" title="基础知识"></a>基础知识</h3><p><strong>命令类型</strong></p><ol><li><strong>管道命令</strong><br>定义：使用管道符（<code>|</code>）将多个简单命令连接起来。<br>特点：相邻两个简单命令中，左边命令的输出作为右边命令的输入。</li><li><strong>简单命令</strong><br>包括以下三种类型：<ol><li><strong>外置命令：</strong><br>定义：Linux系统中自带的可执行文件，通常是C语言编写的程序，如ls、cp等。<br>执行方式：通过调用可执行文件并传入参数，使用execvp函数实现。</li><li><strong>内置命令：</strong><br>定义：Shell内部实现的命令，如cd、exit、help等。<br>特点：与当前Shell进程相关，需要在Shell内部实现，不依赖外部可执行文件。</li><li><strong>重定向命令：</strong><br>定义：对输入或输出进行重定向的命令，包括输入重定向（<、<<）和输出重定向（>、>>）。<br>实现方法：通过操作文件描述符，使用open、close、dup2等系统调用实现重定向。</li></ol></li></ol><p><strong>具体介绍（记录的是我不太了解的概念）</strong></p><ol><li><strong>管道命令</strong><ul><li>管道是数据通道，允许一个进程的输出作为另一个进程的输入。 </li><li>实现步骤：<ol><li>创建管道：使用<code>pipe</code>系统调用，生成两个文件描述符，分别用于读和写。</li><li>创建子进程：使用<code>fork</code>创建子进程，在子进程中进行重定向和命令执行。</li><li>重定向：<ul><li>左边命令的输出重定向到管道的写端。</li><li>右边命令的输入重定向到管道的读端。</li></ul></li><li>关闭管道：在进程结束前关闭管道描述符，防止阻塞。</li><li>等待子进程：在父进程中使用<code>wait</code>等待子进程结束。</li></ol></li></ul></li><li><p><strong>重定向命令</strong></p><ol><li><p><strong>输入重定向 </strong></p><ul><li>符号：<code><</code> 和 <code><<</code></li><li>作用：</li></ul><ol><li><code>cat < input.txt</code>：将<code>input.txt</code>的内容作为<code>cat</code>命令的输入。</li><li><code>cat << EOF</code>：从键盘输入，直到输入<code>EOF</code>，作为<code>cat</code>命令的输入。</li></ol><ul><li>实现代码：</li></ul><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">int</span> fd_input = <span class="built_in">open</span>(<span class="string">"input.txt"</span>, O_RDONLY);</span><br><span class="line"><span class="built_in">close</span>(<span class="number">0</span>);</span><br><span class="line"><span class="built_in">dup2</span>(fd_input, <span class="number">0</span>);</span><br></pre></td></tr></table></figure></li><li><p><strong>输出重定向</strong></p><ul><li>符号：<code>></code> 和 <code>>></code></li><li>作用：</li></ul><ol><li><code>ps -a > output.txt</code>：将<code>ps -a</code>的输出重定向到<code>output.txt</code>，并覆盖原文件内容。</li><li><code>ps -a >> output.txt</code>：将<code>ps -a</code>的输出以追加的形式写入<code>output.txt</code>。</li></ol><ul><li>实现代码：</li></ul><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">int</span> fd_output = <span class="built_in">open</span>(<span class="string">"output.txt"</span>, O_WRONLY | O_CREAT | O_TRUNC, <span class="number">0666</span>);</span><br><span class="line"><span class="built_in">close</span>(<span class="number">1</span>);</span><br><span class="line"><span class="built_in">dup2</span>(fd_output, <span class="number">1</span>);</span><br></pre></td></tr></table></figure></li></ol></li></ol><h3 id="具体实现"><a href="#具体实现" class="headerlink" title="具体实现"></a>具体实现</h3><p><strong>管道的实现</strong>：管道的核心思想是将一个命令的标准输出（STDOUT）作为另一个命令的标准输入（STDIN）。这一过程通过 <code>pipe()</code> 系统调用实现，创建一对文件描述符用于数据流的传递。在我的实现中，我首先通过 <code>fork()</code> 创建子进程，然后根据管道的数量分别设置标准输入输出，并使用 <code>execvp()</code> 来执行每个命令。 </p><p><strong>输入输出重定向的实现</strong>：输入重定向 <code><</code> 和输出重定向 <code>></code> 的实现主要依赖 <code>open()</code> 系统调用，它们分别用来将标准输入或标准输出重定向到指定的文件中。在处理 <code><</code> 和 <code><<</code>（here文档）时，我需要对输入流进行相应的修改，将标准输入从键盘重定向到文件或是终端输入。 </p><p><strong>外部命令的实现：</strong>外部命令的执行是在子进程中进行的。首先，使用 <code>fork()</code> 创建一个新的子进程<strong>。</strong>子进程通过 <code>execvp()</code> 系统调用来执行外部命令。<code>execvp()</code> 会用一个新的程序来替换当前进程的映像，执行指定的命令。</p><p><strong>内部命令的实现：</strong>如 <code>cd</code> 和 <code>exit</code>，这些命令不需要通过 <code>execvp()</code> 执行，而直接在Shell内部处理。尤其是 <code>cd</code> 命令，它需要在父进程中更改工作目录，这就涉及到对进程的工作目录的理解。<code>exit</code> 命令则是退出Shell并结束程序运行。</p><p><strong>流程图</strong></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134021190.jpeg" alt="画板"></p><p>以下是具体实现代码中函数的概述：</p><ul><li><code>print_prompt</code>：负责显示Shell提示符，包含用户名、主机名和当前工作目录。</li><li><code>tokenize_input</code>：将用户输入的命令行分割为tokens，处理空格和引号。</li><li><code>parse_commands</code>：根据管道符<code>|</code>将tokens分割为多个命令。</li><li><code>execute_commands</code>：根据命令数量决定是执行单个命令还是处理管道命令。</li><li><code>execute_single_command</code>：执行单个命令，包括内置命令和外部命令。</li><li><code>handle_redirection</code>：处理命令中的输入和输出重定向。</li><li><code>execute_builtin_command</code>：执行内置命令，如<code>cd</code>，<code>exit</code>和<code>help</code>。</li><li><code>display_help</code>：显示内置命令的帮助信息。</li></ul><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br><span class="line">338</span><br><span class="line">339</span><br><span class="line">340</span><br><span class="line">341</span><br><span class="line">342</span><br><span class="line">343</span><br><span class="line">344</span><br><span class="line">345</span><br><span class="line">346</span><br><span class="line">347</span><br><span class="line">348</span><br><span class="line">349</span><br><span class="line">350</span><br><span class="line">351</span><br><span class="line">352</span><br><span class="line">353</span><br><span class="line">354</span><br><span class="line">355</span><br><span class="line">356</span><br><span class="line">357</span><br><span class="line">358</span><br><span class="line">359</span><br><span class="line">360</span><br><span class="line">361</span><br><span class="line">362</span><br><span class="line">363</span><br><span class="line">364</span><br><span class="line">365</span><br><span class="line">366</span><br><span class="line">367</span><br><span class="line">368</span><br><span class="line">369</span><br><span class="line">370</span><br><span class="line">371</span><br><span class="line">372</span><br><span class="line">373</span><br><span class="line">374</span><br><span class="line">375</span><br><span class="line">376</span><br><span class="line">377</span><br><span class="line">378</span><br><span class="line">379</span><br><span class="line">380</span><br><span class="line">381</span><br><span class="line">382</span><br><span class="line">383</span><br><span class="line">384</span><br><span class="line">385</span><br><span class="line">386</span><br><span class="line">387</span><br><span class="line">388</span><br><span class="line">389</span><br><span class="line">390</span><br><span class="line">391</span><br><span class="line">392</span><br><span class="line">393</span><br><span class="line">394</span><br><span class="line">395</span><br><span class="line">396</span><br><span class="line">397</span><br><span class="line">398</span><br><span class="line">399</span><br><span class="line">400</span><br><span class="line">401</span><br><span class="line">402</span><br><span class="line">403</span><br><span class="line">404</span><br><span class="line">405</span><br><span class="line">406</span><br><span class="line">407</span><br><span class="line">408</span><br><span class="line">409</span><br><span class="line">410</span><br><span class="line">411</span><br><span class="line">412</span><br><span class="line">413</span><br><span class="line">414</span><br><span class="line">415</span><br><span class="line">416</span><br><span class="line">417</span><br><span class="line">418</span><br><span class="line">419</span><br><span class="line">420</span><br><span class="line">421</span><br><span class="line">422</span><br><span class="line">423</span><br><span class="line">424</span><br><span class="line">425</span><br><span class="line">426</span><br><span class="line">427</span><br><span class="line">428</span><br><span class="line">429</span><br><span class="line">430</span><br><span class="line">431</span><br><span class="line">432</span><br><span class="line">433</span><br><span class="line">434</span><br><span class="line">435</span><br><span class="line">436</span><br><span class="line">437</span><br><span class="line">438</span><br><span class="line">439</span><br><span class="line">440</span><br><span class="line">441</span><br><span class="line">442</span><br><span class="line">443</span><br><span class="line">444</span><br><span class="line">445</span><br><span class="line">446</span><br><span class="line">447</span><br><span class="line">448</span><br><span class="line">449</span><br><span class="line">450</span><br><span class="line">451</span><br><span class="line">452</span><br><span class="line">453</span><br><span class="line">454</span><br><span class="line">455</span><br><span class="line">456</span><br><span class="line">457</span><br><span class="line">458</span><br><span class="line">459</span><br><span class="line">460</span><br><span class="line">461</span><br><span class="line">462</span><br><span class="line">463</span><br><span class="line">464</span><br><span class="line">465</span><br><span class="line">466</span><br><span class="line">467</span><br><span class="line">468</span><br><span class="line">469</span><br><span class="line">470</span><br><span class="line">471</span><br><span class="line">472</span><br><span class="line">473</span><br><span class="line">474</span><br><span class="line">475</span><br><span class="line">476</span><br><span class="line">477</span><br><span class="line">478</span><br><span class="line">479</span><br><span class="line">480</span><br><span class="line">481</span><br><span class="line">482</span><br><span class="line">483</span><br><span class="line">484</span><br><span class="line">485</span><br><span class="line">486</span><br><span class="line">487</span><br><span class="line">488</span><br><span class="line">489</span><br><span class="line">490</span><br><span class="line">491</span><br><span class="line">492</span><br><span class="line">493</span><br><span class="line">494</span><br><span class="line">495</span><br><span class="line">496</span><br><span class="line">497</span><br><span class="line">498</span><br><span class="line">499</span><br><span class="line">500</span><br><span class="line">501</span><br><span class="line">502</span><br><span class="line">503</span><br><span class="line">504</span><br><span class="line">505</span><br><span class="line">506</span><br><span class="line">507</span><br><span class="line">508</span><br><span class="line">509</span><br><span class="line">510</span><br><span class="line">511</span><br><span class="line">512</span><br><span class="line">513</span><br><span class="line">514</span><br><span class="line">515</span><br><span class="line">516</span><br><span class="line">517</span><br><span class="line">518</span><br><span class="line">519</span><br><span class="line">520</span><br><span class="line">521</span><br><span class="line">522</span><br><span class="line">523</span><br><span class="line">524</span><br><span class="line">525</span><br><span class="line">526</span><br><span class="line">527</span><br><span class="line">528</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><iostream></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><vector></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><string></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><unistd.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><fcntl.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><sys/wait.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><pwd.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><cstring></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><cerrno></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">// 使用标准命名空间</span></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> std;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 函数声明</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">print_prompt</span><span class="params">()</span></span>;</span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">tokenize_input</span><span class="params">(<span class="type">const</span> string &input, vector<string> &tokens)</span></span>;</span><br><span class="line">vector<vector<string>> <span class="built_in">parse_commands</span>(<span class="type">const</span> vector<string> &tokens);</span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">execute_commands</span><span class="params">(<span class="type">const</span> vector<vector<string>> &commands)</span></span>;</span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">execute_single_command</span><span class="params">(vector<string> &tokens)</span></span>;</span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">handle_redirection</span><span class="params">(vector<string> &tokens)</span></span>;</span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">execute_builtin_command</span><span class="params">(<span class="type">const</span> vector<string> &tokens)</span></span>;</span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">display_help</span><span class="params">()</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 主函数</span></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">()</span> </span>{</span><br><span class="line"> string input;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">while</span> (<span class="literal">true</span>) {</span><br><span class="line"> <span class="comment">// 显示Shell提示符</span></span><br><span class="line"> <span class="built_in">print_prompt</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 读取用户输入</span></span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">getline</span>(cin, input)) {</span><br><span class="line"> <span class="comment">// 处理EOF (如Ctrl+D)</span></span><br><span class="line"> cout << endl;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 解析输入为tokens</span></span><br><span class="line"> vector<string> tokens;</span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">tokenize_input</span>(input, tokens)) {</span><br><span class="line"> <span class="keyword">continue</span>; <span class="comment">// 如果解析失败，重新显示提示符</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果没有输入命令，继续循环</span></span><br><span class="line"> <span class="keyword">if</span> (tokens.<span class="built_in">empty</span>()) {</span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 解析tokens为多个命令（处理管道）</span></span><br><span class="line"> vector<vector<string>> commands = <span class="built_in">parse_commands</span>(tokens);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 执行解析后的命令</span></span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">execute_commands</span>(commands)) {</span><br><span class="line"> <span class="keyword">break</span>; <span class="comment">// 如果执行命令返回false，则退出Shell</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 显示Shell提示符，包括用户名、主机名和当前工作目录。</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">print_prompt</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="type">char</span>* prompt_symbol = <span class="string">"$"</span>;</span><br><span class="line"> <span class="type">uid_t</span> uid = <span class="built_in">getuid</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (uid == <span class="number">0</span>) { <span class="comment">// 判断是否为根用户</span></span><br><span class="line"> prompt_symbol = <span class="string">"#"</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">struct</span> <span class="title class_">passwd</span>* pwd = <span class="built_in">getpwuid</span>(uid);</span><br><span class="line"> <span class="keyword">if</span> (pwd == <span class="literal">nullptr</span>) {</span><br><span class="line"> cout << <span class="string">"$ "</span>;</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">char</span> hostname[<span class="number">128</span>] = {<span class="number">0</span>};</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">gethostname</span>(hostname, <span class="built_in">sizeof</span>(hostname)) != <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">strcpy</span>(hostname, <span class="string">"unknown"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">char</span> cwd[<span class="number">256</span>] = {<span class="number">0</span>};</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">getcwd</span>(cwd, <span class="built_in">sizeof</span>(cwd)) == <span class="literal">nullptr</span>) {</span><br><span class="line"> <span class="built_in">strcpy</span>(cwd, <span class="string">"unknown"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 使用ANSI颜色代码美化提示符</span></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"\033[1;32m%s@%s\033[0m:\033[1;34m%s\033[0m%s "</span>, pwd->pw_name, hostname, cwd, prompt_symbol);</span><br><span class="line"> <span class="built_in">fflush</span>(stdout); <span class="comment">// 确保提示符立即显示</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 将输入字符串分割为tokens，基于空格分隔。</span></span><br><span class="line"><span class="comment"> * @param input 用户输入的命令行</span></span><br><span class="line"><span class="comment"> * @param tokens 分割后的命令和参数</span></span><br><span class="line"><span class="comment"> * @return 成功返回true，失败返回false</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">tokenize_input</span><span class="params">(<span class="type">const</span> string &input, vector<string> &tokens)</span> </span>{</span><br><span class="line"> tokens.<span class="built_in">clear</span>();</span><br><span class="line"> string token;</span><br><span class="line"> <span class="type">bool</span> in_quotes = <span class="literal">false</span>;</span><br><span class="line"> <span class="type">char</span> quote_char = <span class="string">'\0'</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">size_t</span> i = <span class="number">0</span>; i < input.<span class="built_in">length</span>(); ++i) {</span><br><span class="line"> <span class="type">char</span> c = input[i];</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (in_quotes) {</span><br><span class="line"> <span class="keyword">if</span> (c == quote_char) {</span><br><span class="line"> in_quotes = <span class="literal">false</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> token += c;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (c == <span class="string">'\''</span> || c == <span class="string">'\"'</span>) {</span><br><span class="line"> in_quotes = <span class="literal">true</span>;</span><br><span class="line"> quote_char = c;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (<span class="built_in">isspace</span>(c)) {</span><br><span class="line"> <span class="keyword">if</span> (!token.<span class="built_in">empty</span>()) {</span><br><span class="line"> tokens.<span class="built_in">push_back</span>(token);</span><br><span class="line"> token.<span class="built_in">clear</span>();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (c == <span class="string">'|'</span> || c == <span class="string">'<'</span> || c == <span class="string">'>'</span>) {</span><br><span class="line"> <span class="keyword">if</span> (!token.<span class="built_in">empty</span>()) {</span><br><span class="line"> tokens.<span class="built_in">push_back</span>(token);</span><br><span class="line"> token.<span class="built_in">clear</span>();</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 处理双字符符号 << 和 >></span></span><br><span class="line"> <span class="keyword">if</span> ((c == <span class="string">'<'</span> || c == <span class="string">'>'</span>) && i + <span class="number">1</span> < input.<span class="built_in">length</span>() && input[i + <span class="number">1</span>] == c) {</span><br><span class="line"> tokens.<span class="built_in">emplace_back</span>(<span class="built_in">string</span>(<span class="number">2</span>, c));</span><br><span class="line"> i++; <span class="comment">// 跳过下一个字符</span></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> tokens.<span class="built_in">emplace_back</span>(<span class="built_in">string</span>(<span class="number">1</span>, c));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> token += c;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (in_quotes) {</span><br><span class="line"> cerr << <span class="string">"Error: Mismatched quotes in input."</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!token.<span class="built_in">empty</span>()) {</span><br><span class="line"> tokens.<span class="built_in">push_back</span>(token);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 将tokens分割为多个命令，以处理管道。</span></span><br><span class="line"><span class="comment"> * @param tokens 分割后的命令和参数</span></span><br><span class="line"><span class="comment"> * @return 分割后的命令集合</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line">vector<vector<string>> <span class="built_in">parse_commands</span>(<span class="type">const</span> vector<string> &tokens) {</span><br><span class="line"> vector<vector<string>> commands;</span><br><span class="line"> vector<string> current_command;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">const</span> <span class="keyword">auto</span> &token : tokens) {</span><br><span class="line"> <span class="keyword">if</span> (token == <span class="string">"|"</span>) {</span><br><span class="line"> <span class="keyword">if</span> (current_command.<span class="built_in">empty</span>()) {</span><br><span class="line"> cerr << <span class="string">"Error: Invalid null command."</span> << endl;</span><br><span class="line"> commands.<span class="built_in">clear</span>();</span><br><span class="line"> <span class="keyword">return</span> commands;</span><br><span class="line"> }</span><br><span class="line"> commands.<span class="built_in">push_back</span>(current_command);</span><br><span class="line"> current_command.<span class="built_in">clear</span>();</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> current_command.<span class="built_in">push_back</span>(token);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!current_command.<span class="built_in">empty</span>()) {</span><br><span class="line"> commands.<span class="built_in">push_back</span>(current_command);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> commands;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 执行解析后的命令集合，包括单个命令和带管道的命令。</span></span><br><span class="line"><span class="comment"> * @param commands 分割后的命令集合</span></span><br><span class="line"><span class="comment"> * @return 成功返回true，若执行exit命令则返回false</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">execute_commands</span><span class="params">(<span class="type">const</span> vector<vector<string>> &commands)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (commands.<span class="built_in">empty</span>()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果只有一个命令，直接执行</span></span><br><span class="line"> <span class="keyword">if</span> (commands.<span class="built_in">size</span>() == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">execute_single_command</span>(<span class="keyword">const_cast</span><vector<string> &>(commands[<span class="number">0</span>]));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 处理管道命令</span></span><br><span class="line"> <span class="type">size_t</span> num_commands = commands.<span class="built_in">size</span>();</span><br><span class="line"> vector<<span class="type">int</span>> pipe_fds;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 创建所有需要的管道</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">size_t</span> i = <span class="number">0</span>; i < num_commands - <span class="number">1</span>; ++i) {</span><br><span class="line"> <span class="type">int</span> fd[<span class="number">2</span>];</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">pipe</span>(fd) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"pipe"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> pipe_fds.<span class="built_in">push_back</span>(fd[<span class="number">0</span>]);</span><br><span class="line"> pipe_fds.<span class="built_in">push_back</span>(fd[<span class="number">1</span>]);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 存储所有子进程的PID</span></span><br><span class="line"> vector<<span class="type">pid_t</span>> pids;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">size_t</span> i = <span class="number">0</span>; i < num_commands; ++i) {</span><br><span class="line"> <span class="type">pid_t</span> pid = fork();</span><br><span class="line"> <span class="keyword">if</span> (pid < <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"fork"</span>);</span><br><span class="line"> <span class="comment">// 关闭所有打开的管道</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> fd : pipe_fds) {</span><br><span class="line"> <span class="built_in">close</span>(fd);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (pid == <span class="number">0</span>) { <span class="comment">// 子进程</span></span><br><span class="line"> <span class="comment">// 如果不是第一个命令，重定向标准输入</span></span><br><span class="line"> <span class="keyword">if</span> (i > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">dup2</span>(pipe_fds[(i - <span class="number">1</span>) * <span class="number">2</span>], STDIN_FILENO) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"dup2"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(EXIT_FAILURE);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果不是最后一个命令，重定向标准输出</span></span><br><span class="line"> <span class="keyword">if</span> (i < num_commands - <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">dup2</span>(pipe_fds[i * <span class="number">2</span> + <span class="number">1</span>], STDOUT_FILENO) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"dup2"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(EXIT_FAILURE);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 关闭所有管道文件描述符</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> fd : pipe_fds) {</span><br><span class="line"> <span class="built_in">close</span>(fd);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 执行命令</span></span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">execute_single_command</span>(<span class="keyword">const_cast</span><vector<string> &>(commands[i]))) {</span><br><span class="line"> <span class="built_in">exit</span>(EXIT_FAILURE);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">exit</span>(EXIT_SUCCESS);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> { <span class="comment">// 父进程</span></span><br><span class="line"> pids.<span class="built_in">push_back</span>(pid);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 父进程关闭所有管道文件描述符</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> fd : pipe_fds) {</span><br><span class="line"> <span class="built_in">close</span>(fd);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 父进程等待所有子进程结束</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">pid_t</span> pid : pids) {</span><br><span class="line"> <span class="type">int</span> status;</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">waitpid</span>(pid, &status, <span class="number">0</span>) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"waitpid"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 执行单个命令，包括内置命令和外部命令。</span></span><br><span class="line"><span class="comment"> * @param tokens 命令及其参数</span></span><br><span class="line"><span class="comment"> * @return 若执行exit命令则返回false，其他情况返回true</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">execute_single_command</span><span class="params">(vector<string> &tokens)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (tokens.<span class="built_in">empty</span>()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 检查是否为内置命令</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">execute_builtin_command</span>(tokens)) {</span><br><span class="line"> <span class="comment">// 如果是exit命令，返回false以退出Shell</span></span><br><span class="line"> <span class="keyword">if</span> (tokens[<span class="number">0</span>] == <span class="string">"exit"</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 处理外部命令</span></span><br><span class="line"> <span class="type">pid_t</span> pid = fork();</span><br><span class="line"> <span class="keyword">if</span> (pid < <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"fork"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (pid == <span class="number">0</span>) { <span class="comment">// 子进程</span></span><br><span class="line"> <span class="comment">// 处理重定向</span></span><br><span class="line"> <span class="keyword">if</span> (!<span class="built_in">handle_redirection</span>(tokens)) {</span><br><span class="line"> <span class="built_in">exit</span>(EXIT_FAILURE);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 准备execvp的参数</span></span><br><span class="line"> vector<<span class="type">char</span>*> argv;</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">auto</span> &arg : tokens) {</span><br><span class="line"> argv.<span class="built_in">push_back</span>(<span class="built_in">const_cast</span><<span class="type">char</span>*>(arg.<span class="built_in">c_str</span>()));</span><br><span class="line"> }</span><br><span class="line"> argv.<span class="built_in">push_back</span>(<span class="literal">nullptr</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 执行外部命令</span></span><br><span class="line"> <span class="built_in">execvp</span>(argv[<span class="number">0</span>], argv.<span class="built_in">data</span>());</span><br><span class="line"> <span class="comment">// 如果execvp返回，说明执行失败</span></span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"execvp"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(EXIT_FAILURE);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> { <span class="comment">// 父进程</span></span><br><span class="line"> <span class="comment">// 等待子进程结束</span></span><br><span class="line"> <span class="type">int</span> status;</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">waitpid</span>(pid, &status, <span class="number">0</span>) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"waitpid"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 处理命令中的输入和输出重定向。</span></span><br><span class="line"><span class="comment"> * @param tokens 命令及其参数</span></span><br><span class="line"><span class="comment"> * @return 成功返回true，失败返回false</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">handle_redirection</span><span class="params">(vector<string> &tokens)</span> </span>{</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">size_t</span> i = <span class="number">0</span>; i < tokens.<span class="built_in">size</span>(); ++i) {</span><br><span class="line"> <span class="keyword">if</span> (tokens[i] == <span class="string">"<"</span>) {</span><br><span class="line"> <span class="keyword">if</span> (i + <span class="number">1</span> >= tokens.<span class="built_in">size</span>()) {</span><br><span class="line"> cerr << <span class="string">"Error: No input file specified for redirection."</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 打开输入文件</span></span><br><span class="line"> <span class="type">int</span> fd_in = <span class="built_in">open</span>(tokens[i + <span class="number">1</span>].<span class="built_in">c_str</span>(), O_RDONLY);</span><br><span class="line"> <span class="keyword">if</span> (fd_in == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"open for input redirection"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 重定向标准输入</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">dup2</span>(fd_in, STDIN_FILENO) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"dup2 for input redirection"</span>);</span><br><span class="line"> <span class="built_in">close</span>(fd_in);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">close</span>(fd_in);</span><br><span class="line"> <span class="comment">// 移除重定向符号和文件名</span></span><br><span class="line"> tokens.<span class="built_in">erase</span>(tokens.<span class="built_in">begin</span>() + i, tokens.<span class="built_in">begin</span>() + i + <span class="number">2</span>);</span><br><span class="line"> i--; <span class="comment">// 调整索引</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (tokens[i] == <span class="string">"<<"</span>) {</span><br><span class="line"> <span class="keyword">if</span> (i + <span class="number">1</span> >= tokens.<span class="built_in">size</span>()) {</span><br><span class="line"> cerr << <span class="string">"Error: No delimiter specified for here-document."</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> string delimiter = tokens[i + <span class="number">1</span>];</span><br><span class="line"> string input_data;</span><br><span class="line"> string line;</span><br><span class="line"></span><br><span class="line"> cout << <span class="string">"> "</span>;</span><br><span class="line"> <span class="built_in">fflush</span>(stdout);</span><br><span class="line"> <span class="comment">// 读取直到遇到定界符</span></span><br><span class="line"> <span class="keyword">while</span> (<span class="built_in">getline</span>(cin, line)) {</span><br><span class="line"> <span class="keyword">if</span> (line == delimiter) {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> input_data += line + <span class="string">"\n"</span>;</span><br><span class="line"> cout << <span class="string">"> "</span>;</span><br><span class="line"> <span class="built_in">fflush</span>(stdout);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 创建管道并写入数据</span></span><br><span class="line"> <span class="type">int</span> pipe_fd[<span class="number">2</span>];</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">pipe</span>(pipe_fd) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"pipe for here-document"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">ssize_t</span> bytes_written = <span class="built_in">write</span>(pipe_fd[<span class="number">1</span>], input_data.<span class="built_in">c_str</span>(), input_data.<span class="built_in">size</span>());</span><br><span class="line"> <span class="keyword">if</span> (bytes_written == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"write to pipe for here-document"</span>);</span><br><span class="line"> <span class="built_in">close</span>(pipe_fd[<span class="number">0</span>]);</span><br><span class="line"> <span class="built_in">close</span>(pipe_fd[<span class="number">1</span>]);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">close</span>(pipe_fd[<span class="number">1</span>]); <span class="comment">// 关闭写端</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 重定向标准输入到管道的读端</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">dup2</span>(pipe_fd[<span class="number">0</span>], STDIN_FILENO) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"dup2 for here-document"</span>);</span><br><span class="line"> <span class="built_in">close</span>(pipe_fd[<span class="number">0</span>]);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">close</span>(pipe_fd[<span class="number">0</span>]);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 移除重定向符号和定界符</span></span><br><span class="line"> tokens.<span class="built_in">erase</span>(tokens.<span class="built_in">begin</span>() + i, tokens.<span class="built_in">begin</span>() + i + <span class="number">2</span>);</span><br><span class="line"> i--; <span class="comment">// 调整索引</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (tokens[i] == <span class="string">">"</span>) {</span><br><span class="line"> <span class="keyword">if</span> (i + <span class="number">1</span> >= tokens.<span class="built_in">size</span>()) {</span><br><span class="line"> cerr << <span class="string">"Error: No output file specified for redirection."</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 打开输出文件（截断模式）</span></span><br><span class="line"> <span class="type">int</span> fd_out = <span class="built_in">open</span>(tokens[i + <span class="number">1</span>].<span class="built_in">c_str</span>(), O_WRONLY | O_CREAT | O_TRUNC, <span class="number">0666</span>);</span><br><span class="line"> <span class="keyword">if</span> (fd_out == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"open for output redirection"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 重定向标准输出</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">dup2</span>(fd_out, STDOUT_FILENO) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"dup2 for output redirection"</span>);</span><br><span class="line"> <span class="built_in">close</span>(fd_out);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">close</span>(fd_out);</span><br><span class="line"> <span class="comment">// 移除重定向符号和文件名</span></span><br><span class="line"> tokens.<span class="built_in">erase</span>(tokens.<span class="built_in">begin</span>() + i, tokens.<span class="built_in">begin</span>() + i + <span class="number">2</span>);</span><br><span class="line"> i--; <span class="comment">// 调整索引</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (tokens[i] == <span class="string">">>"</span>) {</span><br><span class="line"> <span class="keyword">if</span> (i + <span class="number">1</span> >= tokens.<span class="built_in">size</span>()) {</span><br><span class="line"> cerr << <span class="string">"Error: No output file specified for append redirection."</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 打开输出文件（追加模式）</span></span><br><span class="line"> <span class="type">int</span> fd_out = <span class="built_in">open</span>(tokens[i + <span class="number">1</span>].<span class="built_in">c_str</span>(), O_WRONLY | O_CREAT | O_APPEND, <span class="number">0666</span>);</span><br><span class="line"> <span class="keyword">if</span> (fd_out == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"open for append redirection"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 重定向标准输出</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">dup2</span>(fd_out, STDOUT_FILENO) == <span class="number">-1</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"dup2 for append redirection"</span>);</span><br><span class="line"> <span class="built_in">close</span>(fd_out);</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">close</span>(fd_out);</span><br><span class="line"> <span class="comment">// 移除重定向符号和文件名</span></span><br><span class="line"> tokens.<span class="built_in">erase</span>(tokens.<span class="built_in">begin</span>() + i, tokens.<span class="built_in">begin</span>() + i + <span class="number">2</span>);</span><br><span class="line"> i--; <span class="comment">// 调整索引</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 执行内置命令（如cd、exit、help）。</span></span><br><span class="line"><span class="comment"> * @param tokens 命令及其参数</span></span><br><span class="line"><span class="comment"> * @return 如果是内置命令并已执行，返回true；否则返回false</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">bool</span> <span class="title">execute_builtin_command</span><span class="params">(<span class="type">const</span> vector<string> &tokens)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (tokens.<span class="built_in">empty</span>()) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">const</span> string &cmd = tokens[<span class="number">0</span>];</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (cmd == <span class="string">"cd"</span>) {</span><br><span class="line"> <span class="keyword">if</span> (tokens.<span class="built_in">size</span>() > <span class="number">2</span>) {</span><br><span class="line"> cerr << <span class="string">"Usage: cd [directory]"</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">const</span> <span class="type">char</span> *path;</span><br><span class="line"> <span class="keyword">if</span> (tokens.<span class="built_in">size</span>() == <span class="number">1</span>) { <span class="comment">// 没有指定路径，切换到home目录</span></span><br><span class="line"> <span class="keyword">struct</span> <span class="title class_">passwd</span> *pw = <span class="built_in">getpwuid</span>(<span class="built_in">getuid</span>());</span><br><span class="line"> <span class="keyword">if</span> (pw == <span class="literal">nullptr</span>) {</span><br><span class="line"> cerr << <span class="string">"cd: Unable to get home directory."</span> << endl;</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> path = pw->pw_dir;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> { <span class="comment">// 切换到指定路径</span></span><br><span class="line"> path = tokens[<span class="number">1</span>].<span class="built_in">c_str</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">chdir</span>(path) != <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">perror</span>(<span class="string">"cd"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (cmd == <span class="string">"exit"</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>; <span class="comment">// 主函数将处理退出</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span> (cmd == <span class="string">"help"</span>) {</span><br><span class="line"> <span class="built_in">display_help</span>();</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span>; <span class="comment">// 不是内置命令</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 显示内置命令的帮助信息。</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">display_help</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">struct</span> <span class="title class_">passwd</span>* pw = <span class="built_in">getpwuid</span>(<span class="built_in">getuid</span>());</span><br><span class="line"> <span class="type">const</span> <span class="type">char</span>* username = (pw != <span class="literal">nullptr</span>) ? pw->pw_name : <span class="string">"User"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Welcome to the custom shell, %s!\n"</span>, username);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"The shell supports the following built-in commands:\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"1. cd [path] : Change the current working directory to 'path'. If 'path' is omitted, changes to the home directory.\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"2. exit : Exit the shell.\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"3. help : Display this help message.\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Additionally, the shell supports:\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"- Execution of external commands available in the system PATH.\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"- Input redirection using '<' and '<<'.\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"- Output redirection using '>' and '>>'.\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"- Piping between multiple commands using '|'.\n"</span>);</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure><h2 id="实验结果"><a href="#实验结果" class="headerlink" title="实验结果"></a>实验结果</h2><p><strong>内部指令：</strong></p><ol><li><p>help指令</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134103694.png" alt=""></p></li><li><p>cd指令</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134115841.png" alt=""></p></li><li><p>exit指令</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134133292.png" alt=""></p></li></ol><p><strong>外部指令：</strong></p><ol><li><p>ls指令</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134144916.png" alt=""></p></li><li><p>ps指令</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134154541.png" alt=""></p></li><li><p>自行编译后的程序</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134202823.png" alt=""></p></li></ol><p><strong>管道命令：</strong></p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134304652.png" alt=""></p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134318888.png" alt=""></p><p><strong>重定向命令：</strong></p><ol><li><p><code>></code></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134336788.png" alt=""></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134345053.png" alt=""></p></li><li><p><code>>></code></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134350784.png" alt=""></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134355669.png" alt=""></p></li><li><p><code><</code></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134401832.png" alt=""></p></li><li><p><code><<</code></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124134407025.png" alt=""></p></li></ol><h2 id="实验心得"><a href="#实验心得" class="headerlink" title="实验心得"></a>实验心得</h2><p> 在本次实验中，我深入研究了Linux下的管道、重定向以及进程控制等基本操作，结合实际编程实现了一个简易的Shell模拟器。通过编写程序实现命令行的解析、进程管理、输入输出重定向、管道传递等功能，我不仅加深了对操作系统内部机制的理解，还在实际编程中获得了很多宝贵的经验。</p><p> 在实验中，最具挑战性的一部分是对管道和重定向操作的实现。管道符 <code>|</code> 和输入重定向 <code><</code>、输出重定向 <code>></code> 是Shell中最常见的操作，通过它们可以实现命令之间的数据传递与文件输入输出的灵活控制。 这些操作看似简单，但在实现过程中需要注意处理子进程、文件描述符以及异常处理等细节问题。例如，处理输入重定向时，要确保读取到文件内容后关闭文件描述符，避免资源泄漏。 </p>]]></content>
<categories>
<category> 学习笔记 </category>
</categories>
<tags>
<tag> OS </tag>
<tag> 操作系统 </tag>
<tag> 实验报告 </tag>
</tags>
</entry>
<entry>
<title>【SEU-OS-Lab2】东南大学操作系统专题实践二教程</title>
<link href="/2025/01/24/%E3%80%90SEU-OS-Lab2%E3%80%91%E4%B8%9C%E5%8D%97%E5%A4%A7%E5%AD%A6%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E4%B8%93%E9%A2%98%E5%AE%9E%E8%B7%B5%E4%BA%8C%E6%95%99%E7%A8%8B/"/>
<url>/2025/01/24/%E3%80%90SEU-OS-Lab2%E3%80%91%E4%B8%9C%E5%8D%97%E5%A4%A7%E5%AD%A6%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E4%B8%93%E9%A2%98%E5%AE%9E%E8%B7%B5%E4%BA%8C%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<p>本教程区别于其他教程在于，本教程是在WSL2+Linux 6.6.36.6下实现hide系统调用</p><h2 id="实验要求"><a href="#实验要求" class="headerlink" title="实验要求"></a>实验要求</h2><p>实现一个系统调用hide，使得可以根据指定的参数隐藏进程，使用户无法使用ps或top观察到进程状态。</p><p><strong>具体要求：</strong></p><ol><li>实现系统调用int hide(pid_t pid, int on)，在进程pid有效的前提下，如果on置1，进程被隐藏，用户无法通过ps或top观察到进程状态；如果on置0，则恢复正常状态。</li><li>考虑权限问题，只有root用户才能隐藏进程。</li><li>设计一个新的系统调用int hide_user_processes(uid_t uid, char *binname)，参数uid为用户ID号，当binname参数为NULL时，隐藏该用户的所有进程；否则，隐藏二进制映像名为binname的用户进程。该系统调用应与hide系统调用共存。</li><li>在/proc目录下创建一个文件/proc/hidden，该文件可读可写，对应一个全局变量hidden_flag，当hidden_flag为0时，所有进程都无法隐藏，即便此前进程被hide系统调用要求隐藏。只有当hidden_flag为1时，此前通过hide调用要求被屏蔽的进程才隐藏起来。</li><li>在/proc目录下创建一个文件/proc/hidden_process，该文件的内容包含所有被隐藏进程的pid，各pid之间用空格分开。</li></ol><h2 id="实验目的"><a href="#实验目的" class="headerlink" title="实验目的"></a>实验目的</h2><ul><li>加深理解进程控制块、进程队列等概念，了解进程管理的具体实施方法。</li><li>了解系统调用的实现原理，掌握Linux内核相关接口，实现自己的系统调用。</li></ul><h2 id="实验环境"><a href="#实验环境" class="headerlink" title="实验环境"></a>实验环境</h2><ul><li>WSL2</li><li>Ubuntu22.04</li><li>Linux版本：Linux 6.6.36.6-microsoft-standard-WSL2</li></ul><h2 id="前置知识"><a href="#前置知识" class="headerlink" title="前置知识"></a>前置知识</h2><p>本实验涉及到对WSL系统内核的重新编译和替换，可参考以下教程</p><p><a href="https://zhuanlan.zhihu.com/p/18715471543">WSL2更换Linux Kernel为6.6.36.6版本 - 知乎</a></p><h2 id="实验内容"><a href="#实验内容" class="headerlink" title="实验内容"></a>实验内容</h2><h3 id="实现hide和hide-user-processes系统调用"><a href="#实现hide和hide-user-processes系统调用" class="headerlink" title="实现hide和hide_user_processes系统调用"></a>实现hide和hide_user_processes系统调用</h3><p>在此部分，我将实现实验要求中的1，2，3。即实现hide和hide_user_processes系统调用，使得用户可以通过调用这两个系统调用来控制进程的隐藏，并增加验权流程。</p><h4 id="实现思路"><a href="#实现思路" class="headerlink" title="实现思路"></a>实现思路</h4><p>为了实现进程隐藏功能，我们需要了解 <code>ps</code> 和 <code>top</code> 等命令是如何获取进程信息的。这些命令实际上是通过读取 <code>/proc</code> 文件系统中的虚拟文件来获取进程信息的。每个进程在 <code>/proc</code> 中都有一个以其 PID 命名的目录，这些目录中包含了进程的 PCB（进程控制块）信息。</p><p>因此，我们可以通过在构建 <code>/proc</code> 文件系统时增加一个标志位 <code>hidden</code> 来实现进程隐藏。当系统检测到某个进程的 <code>hidden</code> 标志位为 1 时，就不在 <code>/proc</code> 中为其构建对应的目录。</p><h4 id="流程图"><a href="#流程图" class="headerlink" title="流程图"></a>流程图</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132547927.jpeg" alt="画板"></p><p><strong>具体实现步骤如下：</strong></p><ol><li><p><strong>修改<code>task_struct</code>结构体：</strong></p><ul><li><p>在 <code>task_struct</code>（进程控制块）中添加一个 <code>hidden</code> 标志位，用于标记进程是否被隐藏。</p><ul><li>即在 <code>include/linux/sched.h</code> 文件中的<code>struct task_struct{}</code>结构体中按如下添加代码。用户自定义的参数必须添加在注释预留位与<code>randomized_struct_fields_end</code>之间。</li></ul><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * New fields for task_struct should be added above here, so that</span></span><br><span class="line"><span class="comment"> * they are included in the randomized portion of task_struct.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="type">int</span> hidden; <span class="comment">// 0:unhidden; 1:hidden</span></span><br><span class="line">randomized_struct_fields_end</span><br></pre></td></tr></table></figure></li></ul></li><li><p><strong>初始化<code>hidden</code> 标志位： </strong></p><ul><li><p>我们需要在进程创建的时候对这个标志位进行初始化，而进程的创建又是通过<code>fork</code>调用实现的。</p><ul><li>即在 <code>kernel/fork.c</code> 文件中的 <code>copy_process</code> 函数中按如下添加代码，进行 <code>hidden</code> 的初始化。</li></ul><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">__latent_entropy <span class="keyword">struct</span> task_struct *<span class="title function_">copy_process</span><span class="params">(<span class="keyword">struct</span> pid *pid,</span></span><br><span class="line"><span class="params"> <span class="type">int</span> trace,</span></span><br><span class="line"><span class="params"> <span class="type">int</span> node,</span></span><br><span class="line"><span class="params"> <span class="keyword">struct</span> kernel_clone_args *args)</span></span><br><span class="line">{</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line"> p = dup_task_struct(current, node);</span><br><span class="line"> <span class="keyword">if</span> (!p)</span><br><span class="line"> <span class="keyword">goto</span> fork_out;</span><br><span class="line"> <span class="comment">// 代码必须要放在这之后，因为上面的代码是对进程的task_struct *p进行初始化</span></span><br><span class="line"> p -> hidden = <span class="number">0</span>; <span class="comment">// add code</span></span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li><li><p><strong>实现 <code>hide</code> 系统调用：</strong></p><ul><li>创建一个新的系统调用 <code>SYSCALL_DEFINE2(hide, pid_t, pid, int, on)</code>。</li><li>在该系统调用中，通过 <code>uid_eq(current_euid(), GLOBAL_ROOT_UID)</code>，确保只有 root 用户可以隐藏进程。</li><li>通过 <code>pid_task(find_vpid(pid), PIDTYPE_PID)</code> 查找目标进程。</li><li><p>根据 <code>on</code> 参数设置目标进程的 <code>hidden</code> 标志位。</p><ul><li>即在 <code>kernel/sys.c</code> 文件末尾添加如下代码</li></ul><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 接受一个 pid 和 on 参数，根据 on 参数设置pid对应的进程的hidden标志位</span></span><br><span class="line">SYSCALL_DEFINE2(hide, <span class="type">pid_t</span>, pid, <span class="type">int</span>, on)</span><br><span class="line">{</span><br><span class="line"> <span class="keyword">if</span> (pid < <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> -EINVAL;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> printk(<span class="string">"hide process invoked with params: pid=%d on=%d\n"</span>, pid, on);</span><br><span class="line"> printk(<span class="string">"current uid = %d\n"</span>, get_current_cred()->uid.val);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 校验是否为root用户 </span></span><br><span class="line"> <span class="comment">// ps:还可使用capable(CAP_SYS_ADMIN)等来检测用户权限，而不是限制为root用户</span></span><br><span class="line"> <span class="keyword">if</span>(!uid_eq(current_euid(), GLOBAL_ROOT_UID)) {</span><br><span class="line"> printk(<span class="string">"you aren't the root user! try to use sudo!\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> -EPERM;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 查找目标进程的 task_struct </span></span><br><span class="line"> <span class="comment">// ps:还可使用如 find_task_by_vpid(pid), find_get_task_by_vpid(pid)来查找</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">task_struct</span> *<span class="title">goal_task</span> =</span> pid_task(find_vpid(pid), PIDTYPE_PID);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(goal_task) {</span><br><span class="line"> printk(<span class="string">"goal_task current hidden value: %d\n"</span>, goal_task->hidden);</span><br><span class="line"> goal_task->hidden = on;</span><br><span class="line"> printk(<span class="string">"Set goal_task hidden to %d\n"</span>, goal_task->hidden);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> printk(<span class="string">"Process with pid %d not found\n"</span>, pid);</span><br><span class="line"> <span class="keyword">return</span> -ESRCH;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> printk(<span class="string">"nice system call! goodbye!\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li><li><p><strong>实现<code>hide_user_processes</code>系统调用：</strong></p><ul><li>创建一个新的系统调用 <code>SYSCALL_DEFINE2(hide_user_process, uid_t, uid, char __user *, binname)</code>。</li><li>在该系统调用中，通过 <code>uid_eq(current_euid(), GLOBAL_ROOT_UID)</code>，确保只有 root 用户可以隐藏进程。</li><li>使用<code>for_each_process</code>遍历系统中的所有进程，匹配 <code>uid</code> 和 <code>binname</code>。</li><li><p>对匹配的进程设置 <code>hidden</code> 标志位为1。</p><ul><li>即在 <code>kernel/sys.c</code> 文件末尾添加如下代码</li></ul><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 传递一个用户 ID (uid) 和可选的可执行文件名 (binname)，隐藏指定用户的进程</span></span><br><span class="line">SYSCALL_DEFINE2(hide_user_process, <span class="type">uid_t</span>, uid, <span class="type">char</span> __user *, binname) {</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">task_struct</span> *<span class="title">p</span>;</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">user_namespace</span> *<span class="title">ns</span> =</span> current_user_ns();</span><br><span class="line"> <span class="comment">// TASK_COMM_LEN = 16，task_struct存储的进程名称最长只保留16位</span></span><br><span class="line"> <span class="type">char</span> task_name[TASK_COMM_LEN] = {<span class="number">0</span>}; <span class="comment">// 存储当前进程名</span></span><br><span class="line"> <span class="type">char</span> target_name[TASK_COMM_LEN] = {<span class="number">0</span>}; <span class="comment">// 存储目标进程名</span></span><br><span class="line"> <span class="type">int</span> hide;</span><br><span class="line"> <span class="type">kuid_t</span> kuid;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (uid < <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">return</span> -EINVAL;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> kuid = make_kuid(ns, uid);</span><br><span class="line"> printk(<span class="string">"hide user_process invoked with params: kuid=%d binname=%s\n"</span>, uid, binname);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 检查权限</span></span><br><span class="line"> <span class="keyword">if</span> (!uid_eq(current_euid(), GLOBAL_ROOT_UID)) {</span><br><span class="line"> printk(<span class="string">"you aren't the root user! try to use sudo!\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> -EPERM;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 复制目标进程名到target_name</span></span><br><span class="line"> <span class="keyword">if</span> (binname != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">// 先检查用户空间指针是否可访问</span></span><br><span class="line"> <span class="keyword">if</span> (!access_ok(binname, TASK_COMM_LEN<span class="number">-1</span>)) {</span><br><span class="line"> printk(<span class="string">"Invalid user space pointer\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> -EFAULT;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 复制字符串，最多复制 TASK_COMM_LEN-1 个字符</span></span><br><span class="line"> <span class="keyword">if</span> (strncpy_from_user(target_name, binname, TASK_COMM_LEN<span class="number">-1</span>) < <span class="number">0</span>) {</span><br><span class="line"> printk(<span class="string">"Failed to copy process name from user space\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> -EFAULT;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 确保字符串以 null 结尾</span></span><br><span class="line"> target_name[TASK_COMM_LEN<span class="number">-1</span>] = <span class="string">'\0'</span>;</span><br><span class="line"></span><br><span class="line"> printk(<span class="string">"kernel space target_name = %s\n"</span>, target_name);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> printk(<span class="string">"kernel space target_name = NULL\n"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 遍历进程</span></span><br><span class="line"> for_each_process(p) {</span><br><span class="line"> <span class="keyword">if</span> (uid_eq(task_uid(p), kuid) && task_pid_vnr(p) != <span class="number">0</span>) {</span><br><span class="line"> hide = <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 获取当前进程名</span></span><br><span class="line"> get_task_comm(task_name, p);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果指定了进程名，则进行比较</span></span><br><span class="line"> <span class="keyword">if</span> (binname != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">strcmp</span>(task_name, target_name) != <span class="number">0</span>) {</span><br><span class="line"> hide = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> printk(<span class="string">"scan on '%s' hide it? =%d"</span>, task_name, hide);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (hide) {</span><br><span class="line"> p->hidden = <span class="number">1</span>; <span class="comment">// 对当前进程进行隐藏</span></span><br><span class="line"> printk(<span class="string">"uid = %d, hide pid = %d, process name = %s\n"</span>, </span><br><span class="line"> from_kuid(&init_user_ns, task_uid(p)),</span><br><span class="line"> task_pid_vnr(p), </span><br><span class="line"> task_name);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> printk(<span class="string">"nice system call! goodbye!\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li><li><strong>修改<code>/proc</code>文件系统：</strong><ul><li>在 <code>/proc</code> 文件系统的读取函数中，检查每个进程的 <code>hidden</code> 标志位。如果某个进程的 <code>hidden</code> 标志位为 1，则跳过该进程，不在 <code>/proc</code> 中为其创建目录。<ul><li>即在 <code>fs/proc/base.c</code> 文件中的 <code>proc_pid_readdir</code> 函数中，按如下添加代码，在循环构建 <code>/proc</code> 文件系统时，当检测到进程的标志位 <code>hidden==1</code> 就跳过它的构建，这样该进程就被隐藏啦。<figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">int</span> <span class="title function_">proc_pid_readdir</span><span class="params">(<span class="keyword">struct</span> file *file, <span class="keyword">struct</span> dir_context *ctx)</span></span><br><span class="line">{</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line"> <span class="keyword">for</span> (iter = next_tgid(ns, iter);</span><br><span class="line"> iter.task;</span><br><span class="line"> iter.tgid += <span class="number">1</span>, iter = next_tgid(ns, iter))</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span>(iter.task->hidden==<span class="number">1</span>) <span class="keyword">continue</span>; <span class="comment">//add code</span></span><br><span class="line"> <span class="type">char</span> name[<span class="number">10</span> + <span class="number">1</span>];</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> len;</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li></ul></li><li><p><strong>添加系统调用接口到内核文件中</strong>：</p><ul><li><p>在 <code>include/linux/syscalls.h</code> 文件中添加如下代码，与其他系统调用放在一起。</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">asmlinkage <span class="type">long</span> <span class="title function_">sys_hide</span><span class="params">( <span class="type">pid_t</span> pid, <span class="type">int</span> on )</span>; <span class="comment">//my system call</span></span><br><span class="line">asmlinkage <span class="type">long</span> <span class="title function_">sys_hide_user_process</span><span class="params">( <span class="type">uid_t</span> uid, <span class="type">char</span> __user * binname )</span>; <span class="comment">//my system call</span></span><br></pre></td></tr></table></figure></li><li><p>在 <code>include/uapi/asm-generic/unistd.h</code> 文件中添加系统调用编号，该编号只需要加在64位系统调用末尾，与其他编号不同即可（编号大于512的是留给32位系统调用的），记得改最后的末尾系统调用编号。</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">before</span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_fchmodat2 452</span></span><br><span class="line">__SYSCALL(__NR_fchmodat2, sys_fchmodat2)</span><br><span class="line"><span class="meta">#<span class="keyword">undef</span> __NR_syscalls</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_syscalls 454 <span class="comment">// 末尾系统调用编号</span></span></span><br><span class="line"></span><br><span class="line">after</span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_fchmodat2 452</span></span><br><span class="line">__SYSCALL(__NR_fchmodat2, sys_fchmodat2)</span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_hide 454</span></span><br><span class="line">__SYSCALL(__NR_hide, sys_hide)</span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_hide_user_process 455</span></span><br><span class="line">__SYSCALL(__NR_hide_user_process, sys_hide_user_process)</span><br><span class="line"><span class="meta">#<span class="keyword">undef</span> __NR_syscalls</span></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_syscalls 456 <span class="comment">// 末尾系统调用编号</span></span></span><br></pre></td></tr></table></figure></li><li><p>在对应系统版本的系统调用表中添加如下代码（例如我的是wsl内核，编译后的内核在x86目录下说明编译的是x86版本linux内核，因此我在 <code>arch/x86/entry/syscalls/syscall_64.tbl</code> 文件中添加），注意系统调用编号与上面代码中定义的相同。</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="number">454</span> <span class="number">64</span> hide sys_hide</span><br><span class="line"><span class="number">455</span> <span class="number">64</span> hide_user_process sys_hide_user_process</span><br></pre></td></tr></table></figure></li></ul></li></ol><h3 id="实现-proc-hidden和-proc-hidden-process文件控制"><a href="#实现-proc-hidden和-proc-hidden-process文件控制" class="headerlink" title="实现/proc/hidden和/proc/hidden_process文件控制"></a>实现/proc/hidden和/proc/hidden_process文件控制</h3><p>在此部分，我将实现实验要求中的4，5。即实现 <code>/proc/hidden</code> 和 <code>/proc/hidden_process</code> 文件控制，使得用户可以通过 <code>hidden</code> 文件开关隐藏功能，通过 <code>hidden_process</code> 文件获取所有隐藏进程的pid。</p><h4 id="实现思路-1"><a href="#实现思路-1" class="headerlink" title="实现思路"></a>实现思路</h4><p>要想在 <code>/proc</code> 文件系统中添加文件，我们需要在 <code>fs/proc/root.c</code> 文件中的 <code>proc_root_init</code> 函数中去添加相关代码，让 <code>/proc</code> 文件系统初始化时，能自动创建 <code>hidden</code> 和 <code>hidden_process</code> 文件。</p><p><code>hidden</code> 文件中就保存着 <code>hidden_flag</code> 标志位，对 <code>hidden</code> 文件的读写就是对 <code>hidden_flag</code> 的读写。将 <code>hidden_flag</code> 标志位与 <code>hidden</code> 标志位共同作用到<code>fs/proc/base.c</code> 文件中的 <code>proc_pid_readdir</code> 函数中即可实现整体进程隐藏功能的开关。</p><p><code>hidden_process</code> 文件就作为一个函数的接口，读取该文件时就执行这个函数。该函数即遍历所有进程并输出 <code>hidden</code> 标志位为1的进程的pid。</p><p><strong>对于<code>hidden</code>文件：</strong></p><ol><li><p><strong>创建一个所有内核文件都能获取的hidden_flag标志位</strong>：</p><ul><li><p>可以在 <code>fs/proc</code> 目录下新建一个 <code>var.h</code> 文件</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">extern</span> <span class="type">int</span> hidden_flag;</span><br></pre></td></tr></table></figure></li><li><p>在需要使用 <code>hidden_flag</code> 标志位的文件（ <code>fs/proc/root.c</code> 和 <code>fs/proc/base.c</code> ）中添加头文件引入</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string">"var.h"</span></span></span><br></pre></td></tr></table></figure></li></ul></li><li><p><strong>在 <code>proc_pid_readdir</code> 函数中添加hidden文件初始化的代码：</strong></p><ul><li><p>即在 <code>fs/proc/root.c</code> 文件中添加如下代码</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 获取全局变量</span></span><br><span class="line"><span class="type">int</span> hidden_flag = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 添加读写回调函数</span></span><br><span class="line"><span class="type">static</span> <span class="type">ssize_t</span> <span class="title function_">hidden_read</span><span class="params">(<span class="keyword">struct</span> file *file, <span class="type">char</span> __user *buf,</span></span><br><span class="line"><span class="params"> <span class="type">size_t</span> count, <span class="type">loff_t</span> *ppos)</span></span><br><span class="line">{</span><br><span class="line"> <span class="type">char</span> tmp[<span class="number">32</span>];</span><br><span class="line"> <span class="type">int</span> len;</span><br><span class="line"> <span class="comment">// 如果已经读取过，返回0表示结束</span></span><br><span class="line"> <span class="keyword">if</span> (*ppos > <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> <span class="comment">// 将hidden_flag转换为字符串</span></span><br><span class="line"> len = <span class="built_in">snprintf</span>(tmp, <span class="keyword">sizeof</span>(tmp), <span class="string">"%d\n"</span>, hidden_flag);</span><br><span class="line"> <span class="comment">// 确保不会超出用户提供的缓冲区</span></span><br><span class="line"> <span class="keyword">if</span> (count < len)</span><br><span class="line"> <span class="keyword">return</span> -EINVAL;</span><br><span class="line"> <span class="comment">// 复制到用户空间</span></span><br><span class="line"> <span class="keyword">if</span> (copy_to_user(buf, tmp, len))</span><br><span class="line"> <span class="keyword">return</span> -EFAULT;</span><br><span class="line"> *ppos = len;</span><br><span class="line"> <span class="keyword">return</span> len;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="type">static</span> <span class="type">ssize_t</span> <span class="title function_">hidden_write</span><span class="params">(<span class="keyword">struct</span> file *file, <span class="type">const</span> <span class="type">char</span> __user *buf,</span></span><br><span class="line"><span class="params"> <span class="type">size_t</span> count, <span class="type">loff_t</span> *ppos)</span></span><br><span class="line">{</span><br><span class="line"> <span class="type">char</span> tmp[<span class="number">32</span>];</span><br><span class="line"> <span class="type">int</span> new_value;</span><br><span class="line"> <span class="comment">// 防止缓冲区溢出</span></span><br><span class="line"> <span class="keyword">if</span> (count >= <span class="keyword">sizeof</span>(tmp))</span><br><span class="line"> <span class="keyword">return</span> -EINVAL;</span><br><span class="line"> <span class="comment">// 从用户空间复制数据</span></span><br><span class="line"> <span class="keyword">if</span> (copy_from_user(tmp, buf, count))</span><br><span class="line"> <span class="keyword">return</span> -EFAULT;</span><br><span class="line"> <span class="comment">// 确保字符串结束</span></span><br><span class="line"> tmp[count] = <span class="string">'\0'</span>;</span><br><span class="line"> <span class="comment">// 将字符串转换为整数</span></span><br><span class="line"> <span class="keyword">if</span> (kstrtoint(tmp, <span class="number">10</span>, &new_value) < <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">return</span> -EINVAL;</span><br><span class="line"> <span class="comment">// 更新全局变量</span></span><br><span class="line"> hidden_flag = new_value;</span><br><span class="line"> <span class="keyword">return</span> count;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="type">static</span> <span class="type">const</span> <span class="class"><span class="keyword">struct</span> <span class="title">proc_ops</span> <span class="title">hidden_fops</span> =</span> {</span><br><span class="line"> .proc_read = hidden_read, <span class="comment">// 调用了上面的回调函数</span></span><br><span class="line"> .proc_write = hidden_write,</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="comment">//...exist code...</span></span><br><span class="line"><span class="type">void</span> __init <span class="title function_">proc_root_init</span><span class="params">(<span class="type">void</span>)</span>{</span><br><span class="line"> <span class="comment">//...exist code...</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p>在<code>proc_pid_readdir</code> 函数末尾添加以下代码创建 <code>hidden</code> 文件</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">void</span> __init <span class="title function_">proc_root_init</span><span class="params">(<span class="type">void</span>)</span>{</span><br><span class="line"> <span class="comment">//...exist code...</span></span><br><span class="line"> <span class="comment">// 创建 /proc/hidden 文件</span></span><br><span class="line"> <span class="keyword">if</span>(!proc_create(<span class="string">"hidden"</span>, <span class="number">0600</span>, <span class="literal">NULL</span>, &hidden_fops)) {</span><br><span class="line"> pr_err(<span class="string">"Failed to create /proc/hidden\n"</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li><li><p><strong>修改 <code>/proc</code> 文件系统：</strong></p><ul><li><p>在<code>fs/proc/base.c</code> 文件中的 <code>proc_pid_readdir</code> 函数中不仅加入对 <code>hidden</code> 标志位的判断，还要加入对 <code>hidden_flag</code> 标志位的判断。</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">int</span> <span class="title function_">proc_pid_readdir</span><span class="params">(<span class="keyword">struct</span> file *file, <span class="keyword">struct</span> dir_context *ctx)</span></span><br><span class="line">{</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line"> <span class="keyword">for</span> (iter = next_tgid(ns, iter);</span><br><span class="line"> iter.task;</span><br><span class="line"> iter.tgid += <span class="number">1</span>, iter = next_tgid(ns, iter))</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span>(hidden_flag==<span class="number">0</span> && iter.task->hidden==<span class="number">1</span>) <span class="keyword">continue</span>; <span class="comment">//add code</span></span><br><span class="line"> <span class="type">char</span> name[<span class="number">10</span> + <span class="number">1</span>];</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> len;</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// ...exist code...</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li></ol><p><strong>对于 <code>hidden_process</code>文件：</strong></p><ol><li><p><strong>在 <code>proc_pid_readdir</code>函数中添加 <code>hidden_process</code> 文件初始化的代码：</strong></p><ul><li><p>即在 <code>fs/proc/root.c</code> 文件中添加如下代码</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// hidden_process的读回调</span></span><br><span class="line"><span class="type">static</span> <span class="type">ssize_t</span> <span class="title function_">hidden_process_read</span><span class="params">(<span class="keyword">struct</span> file *file, <span class="type">char</span> __user *buf,</span></span><br><span class="line"><span class="params"> <span class="type">size_t</span> count, <span class="type">loff_t</span> *ppos)</span> {</span><br><span class="line"> <span class="type">static</span> <span class="type">char</span> kbuf[<span class="number">1024</span>*<span class="number">8</span>]=<span class="string">""</span>; </span><br><span class="line"> <span class="type">char</span> tmp[<span class="number">128</span>];</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">task_struct</span> *<span class="title">p</span>;</span></span><br><span class="line"> <span class="type">int</span> len;</span><br><span class="line"> <span class="type">pid_t</span> pid;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (*ppos > <span class="number">0</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 清空缓冲区</span></span><br><span class="line"> <span class="built_in">memset</span>(kbuf, <span class="number">0</span>, <span class="keyword">sizeof</span>(kbuf)); </span><br><span class="line"></span><br><span class="line"> <span class="comment">// 遍历进程并处理</span></span><br><span class="line"> for_each_process(p) {</span><br><span class="line"> <span class="comment">// 获取被隐藏的进程，这里只记录pid不为0的进程，因为pid=0的进程原本是看不到的</span></span><br><span class="line"> <span class="keyword">if</span> (p->hidden == <span class="number">1</span> && task_pid_vnr(p)!=<span class="number">0</span>) {</span><br><span class="line"> pid = task_pid_vnr(p); </span><br><span class="line"> <span class="built_in">snprintf</span>(tmp, <span class="keyword">sizeof</span>(tmp), <span class="string">"%d "</span>, pid);</span><br><span class="line"> <span class="built_in">strcat</span>(kbuf, tmp);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> len = <span class="built_in">strlen</span>(kbuf);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 将内核空间的数据复制到用户空间</span></span><br><span class="line"> <span class="keyword">if</span> (copy_to_user(buf, kbuf, len))</span><br><span class="line"> <span class="keyword">return</span> -EFAULT;</span><br><span class="line"></span><br><span class="line"> *ppos += len; <span class="comment">// 更新文件位置</span></span><br><span class="line"> <span class="keyword">return</span> len;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="type">static</span> <span class="type">const</span> <span class="class"><span class="keyword">struct</span> <span class="title">proc_ops</span> <span class="title">hidden_process_fops</span> =</span> {</span><br><span class="line"> .proc_read = hidden_process_read,</span><br><span class="line">};</span><br><span class="line"></span><br></pre></td></tr></table></figure></li><li><p>在<code>proc_pid_readdir</code> 函数末尾添加以下代码创建 <code>hidden_process</code> 文件</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">void</span> __init <span class="title function_">proc_root_init</span><span class="params">(<span class="type">void</span>)</span>{</span><br><span class="line"> <span class="comment">//...exist code...</span></span><br><span class="line"> <span class="comment">// 创建 /proc/hidden_process 文件</span></span><br><span class="line"> <span class="keyword">if</span>(!proc_create(<span class="string">"hidden_process"</span>, <span class="number">0600</span>, <span class="literal">NULL</span>, &hidden_process_fops)) {</span><br><span class="line"> pr_err(<span class="string">"Failed to create /proc/hidden_process\n"</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li></ul></li></ol><h2 id="实验结果"><a href="#实验结果" class="headerlink" title="实验结果"></a>实验结果</h2><p>测试函数 <code>test_hide.c</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><unistd.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><sys/syscall.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_hide 454</span></span><br><span class="line"><span class="type">int</span> <span class="title function_">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> {</span><br><span class="line"> <span class="keyword">if</span> (argc != <span class="number">3</span>) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"参数为2个: pid, on\n"</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="type">int</span> pid = atoi(argv[<span class="number">1</span>]);</span><br><span class="line"> <span class="type">int</span> on = atoi(argv[<span class="number">2</span>]);</span><br><span class="line"> <span class="type">long</span> result = syscall(__NR_hide, pid, on);</span><br><span class="line"> <span class="keyword">if</span> (result == <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Successfully called sys_hide\n"</span>);</span><br><span class="line"> } </span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"sys_hide call failed with error code: %ld\n"</span>, result);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li><p><strong>隐藏pid为1的进程</strong></p><p> before</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132609681.png" alt=""></p><p> 有pid=1的进程</p><p> after</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132615201.png" alt=""></p><p> 无pid=1的进程</p><p> 验权</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132620368.png" alt=""></p><p> 非root用户无法调用该系统调用<br> 测试函数 <code>test_hide_user.c</code></p> <figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><unistd.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><sys/syscall.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> __NR_hide_user_process 455 </span></span><br><span class="line"><span class="type">int</span> <span class="title function_">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> {</span><br><span class="line"> <span class="keyword">if</span> (argc != <span class="number">2</span> && argc != <span class="number">3</span>) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Usage:\n"</span>);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">" %s uid # Hide all processes of specified user\n"</span>, argv[<span class="number">0</span>]);</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">" %s uid binname # Hide specific program of specified user\n"</span>, argv[<span class="number">0</span>]);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 解析用户ID</span></span><br><span class="line"> <span class="type">uid_t</span> uid = atoi(argv[<span class="number">1</span>]);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 调用系统调用</span></span><br><span class="line"> <span class="type">long</span> result;</span><br><span class="line"> <span class="keyword">if</span> (argc == <span class="number">2</span>) {</span><br><span class="line"> <span class="comment">// 只隐藏指定用户的所有进程</span></span><br><span class="line"> result = syscall(__NR_hide_user_process, uid, <span class="literal">NULL</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span> <span class="keyword">if</span>(argc == <span class="number">3</span>){</span><br><span class="line"> <span class="comment">// 隐藏指定用户的特定程序</span></span><br><span class="line"> result = syscall(__NR_hide_user_process, uid, argv[<span class="number">2</span>]);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (result == <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Successfully called hide_user_process\n"</span>);</span><br><span class="line"> <span class="keyword">if</span> (argc == <span class="number">2</span>) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Hidden all processes of user %d\n"</span>, uid);</span><br><span class="line"> } </span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"Hidden process '%s' of user %d\n"</span>, argv[<span class="number">2</span>], uid);</span><br><span class="line"> }</span><br><span class="line"> } </span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"System call failed with error code: %ld\n"</span>, result);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure></li><li><p><strong>隐藏warma10032用户的所有进程（uid为1000）</strong></p><p> before</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132626318.png" alt=""></p><p> 有warma10032的进程</p><p> after</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132630983.png" alt=""></p><p> 无warma10032的进程</p><p> 验权</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132635216.png" alt=""></p><p> 非root用户无法调用该系统调用</p></li><li><p><strong>查看<code>/proc/hidden_process</code>获取所有隐藏进程的pid</strong></p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132639140.png" alt=""></p><p> 为之前隐藏的pid=1进程和warma10032用户的进程</p></li><li><p><strong>控制<code>/proc/hidden</code>文件控制开关隐藏功能</strong></p><p> before</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132643116.png" alt=""></p><p> <code>/proc/hidden</code> 为0，pid=1的进程仍然被隐藏</p><p> after</p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132647465.png" alt=""></p><p> <img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124132651418.png" alt=""></p><p> <code>/proc/hidden</code> 置1，被隐藏的进程都出现了</p></li></ol><h2 id="实验心得"><a href="#实验心得" class="headerlink" title="实验心得"></a>实验心得</h2><p>实验如果按部就班的按教程在linux 2.x.x 版本上做应该不难，但因为低版本的虚拟机用起来不习惯，之前也有使用wsl的经验，就打算在wsl上完成本次实验。也许这是一个坏决定，因为教程中的很多方法在新版本的linux上已经不适用了，而且我这还是wsl定制版，在实验中难免踩很多坑。</p><p>在实验中，我觉得最难理解的部分就是linux中进程id的多变，首先linux中有进程和线程之间的包含关系，但它们的底层实现却是相同的task_truct结构体，这就导致在实验过程中要找到真正的进程需要花费一些力气，不仅需要通过task_truct中的pid，tid，pgid，tgid等综合来判断，这些“id们”在不同的namespace中的值也有可能不同。如task_pid_nr和task_pid_vnr两个函数一般会返回两个不同的值，因为一个是内核空间中的pid，一个是当前空间中的pid</p><p>在linux中进行内核实验还有一点难点在于，参数需要在用户空间与内核空间之间进行传递，这也是比较抽象的一点，相同的参数在不同的空间中的值可能并不相同。</p><p>总的来说，修改并编译一个自己的内核，还挺有hacker的感觉。</p>]]></content>
<categories>
<category> 学习笔记 </category>
</categories>
<tags>
<tag> OS </tag>
<tag> 操作系统 </tag>
<tag> 实验报告 </tag>
</tags>
</entry>
<entry>
<title>【SEU-OS-Lab1】东南大学操作系统专题实践一教程</title>
<link href="/2025/01/24/%E3%80%90SEU-OS-Lab1%E3%80%91%E4%B8%9C%E5%8D%97%E5%A4%A7%E5%AD%A6%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E4%B8%93%E9%A2%98%E5%AE%9E%E8%B7%B5%E4%B8%80%E6%95%99%E7%A8%8B/"/>
<url>/2025/01/24/%E3%80%90SEU-OS-Lab1%E3%80%91%E4%B8%9C%E5%8D%97%E5%A4%A7%E5%AD%A6%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E4%B8%93%E9%A2%98%E5%AE%9E%E8%B7%B5%E4%B8%80%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<h3 id="实验内容"><a href="#实验内容" class="headerlink" title="实验内容"></a>实验内容</h3><p>进行Linux环境搭建，熟悉Linux基本操作。</p><ul><li>搭建Linux环境：可选择使用虚拟机（VirtualBox、VMware）或之间安装Linux系统。也可以选择使用wsl便捷搭建Linux环境。</li><li>Linux发行版版本选择：可选择 Fedora 或 Ubuntu。<ul><li>Fedora 是由 Red Hat 社区支持的开源操作系统 ，它由 Fedora 项目社区开发和维护，并得到 Red Hat 的支持。作为一个独立的发行版，Fedora 提供了最新的开源软件和技术，常被用作服务器和开发环境，是许多其他发行版（例如 Red Hat Enterprise Linux，简称 RHEL）的上游版本。</li><li>Ubuntu 是由 Canonical 公司开发的基于 Debian 的 Linux 发行版，首次发布于 2004 年。它以“用户友好性”为核心理念，提供长期支持（LTS）版本，广泛应用于桌面、服务器和云计算领域，是目前最受欢迎的 Linux 发行版之一。</li></ul></li></ul><div class="table-container"><table><thead><tr><th>特性</th><th>Fedora</th><th>Ubuntu</th></tr></thead><tbody><tr><td><strong>目标用户</strong></td><td>开发者、高级用户</td><td>初学者、桌面用户、服务器用户</td></tr><tr><td><strong>稳定性</strong></td><td>较新（测试新技术，有一定不稳定性）</td><td>稳定（LTS 版本长期支持）</td></tr><tr><td><strong>包管理器</strong></td><td>DNF（RPM）</td><td>APT（DEB）</td></tr><tr><td><strong>桌面环境</strong></td><td>默认 GNOME</td><td>默认 GNOME（支持其他版本）</td></tr><tr><td><strong>适用场景</strong></td><td>开发测试、最新技术</td><td>日常使用、云计算、服务器</td></tr><tr><td><strong>社区支持</strong></td><td>专注技术驱动</td><td>用户友好性强，社区支持广泛</td></tr></tbody></table></div><ul><li>搭建C语言开发环境：使用Linux系统中各自的包管理工具安装gcc，g++编译工具</li><li>熟悉Linux基本操作：在本次使用中常用到的Linux指令操作包括文件管理、权限管理、系统管理，如常用的cd，ls，rm，mv，cat，chmod指令。</li></ul><h3 id="实验目的"><a href="#实验目的" class="headerlink" title="实验目的"></a>实验目的</h3><ul><li>熟悉Linux基本环境的搭建</li><li>熟悉C语言编程</li><li>熟悉UNIX的shell/终端/命令行</li><li>学习如何使用适当的代码编辑器，如emacs</li><li>了解UNIX utilities是如何实现的</li></ul><h3 id="实验1——Reverse程序实现"><a href="#实验1——Reverse程序实现" class="headerlink" title="实验1——Reverse程序实现"></a>实验1——Reverse程序实现</h3><p>要求：通过命令行调用一个简单的Reverse程序（C语言）</p><h4 id="设计思路"><a href="#设计思路" class="headerlink" title="设计思路"></a>设计思路</h4><ol><li><p>编写程序</p><ul><li>编写一个 C 程序 <code>reverse.c</code>，从输入文件读取内容并将其按行反向写入输出文件。</li></ul></li><li><p>编译程序</p><ul><li><p>使用 GCC 编译 <code>reverse.c</code>，生成可执行文件 <code>reverse</code>。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc reverse.c -o reverse</span><br></pre></td></tr></table></figure></li></ul></li><li><p>测试程序</p><ul><li><p>使用提供的测试脚本对编译的可执行文件进行测试。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./test-reverse.sh</span><br></pre></td></tr></table></figure></li></ul></li></ol><h4 id="代码流程图"><a href="#代码流程图" class="headerlink" title="代码流程图"></a>代码流程图</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113232822.jpeg" alt="画板"></p><h4 id="源代码"><a href="#源代码" class="headerlink" title="源代码"></a>源代码</h4><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><string.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><sys/stat.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">reverse_lines</span><span class="params">(FILE *input, FILE *output)</span></span>;</span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">print_error_and_exit</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *message, <span class="type">const</span> <span class="type">char</span> *file)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> </span>{</span><br><span class="line"> FILE *input = <span class="literal">NULL</span>;</span><br><span class="line"> FILE *output = <span class="literal">NULL</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 处理命令行参数</span></span><br><span class="line"> <span class="keyword">switch</span> (argc) {</span><br><span class="line"> <span class="keyword">case</span> <span class="number">1</span>: <span class="comment">// 无参数，使用标准输入和标准输出 test 7</span></span><br><span class="line"> <span class="built_in">reverse_lines</span>(stdin, stdout);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> <span class="number">2</span>: <span class="comment">// 一个参数，提供了输入文件</span></span><br><span class="line"> input = <span class="built_in">fopen</span>(argv[<span class="number">1</span>], <span class="string">"r"</span>);</span><br><span class="line"> <span class="keyword">if</span> (!input) {</span><br><span class="line"> <span class="comment">// test 2</span></span><br><span class="line"> <span class="built_in">print_error_and_exit</span>(<span class="string">"reverse: cannot open file"</span>, argv[<span class="number">1</span>]);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">reverse_lines</span>(input, stdout);</span><br><span class="line"> <span class="built_in">fclose</span>(input);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">case</span> <span class="number">3</span>: <span class="comment">// 两个参数，提供了输入和输出文件</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">strcmp</span>(argv[<span class="number">1</span>], argv[<span class="number">2</span>]) == <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">// test 4</span></span><br><span class="line"> <span class="built_in">print_error_and_exit</span>(<span class="string">"reverse: input and output file must differ"</span>, <span class="literal">NULL</span>);</span><br><span class="line"> }</span><br><span class="line"> input = <span class="built_in">fopen</span>(argv[<span class="number">1</span>], <span class="string">"r"</span>);</span><br><span class="line"> <span class="keyword">if</span> (!input) {</span><br><span class="line"> <span class="comment">// test 3</span></span><br><span class="line"> <span class="built_in">print_error_and_exit</span>(<span class="string">"reverse: cannot open file"</span>, argv[<span class="number">1</span>]);</span><br><span class="line"> }</span><br><span class="line"> output = <span class="built_in">fopen</span>(argv[<span class="number">2</span>], <span class="string">"w"</span>);</span><br><span class="line"> <span class="keyword">if</span> (!output) {</span><br><span class="line"> <span class="built_in">fclose</span>(input);</span><br><span class="line"> <span class="built_in">print_error_and_exit</span>(<span class="string">"reverse: cannot open file"</span>, argv[<span class="number">2</span>]);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 获取两个文件指向的元数据</span></span><br><span class="line"> <span class="keyword">struct</span> <span class="title class_">stat</span> stat1, stat2;</span><br><span class="line"> <span class="built_in">stat</span>(argv[<span class="number">1</span>], &stat1);</span><br><span class="line"> <span class="built_in">stat</span>(argv[<span class="number">2</span>], &stat2);</span><br><span class="line"> <span class="comment">// 检查 inode 号和设备号是否相同</span></span><br><span class="line"> <span class="keyword">if</span>(stat<span class="number">1.</span>st_ino == stat<span class="number">2.</span>st_ino && stat<span class="number">1.</span>st_dev == stat<span class="number">2.</span>st_dev){</span><br><span class="line"> <span class="comment">// test 5</span></span><br><span class="line"> <span class="built_in">print_error_and_exit</span>(<span class="string">"reverse: input and output file must differ"</span>, <span class="literal">NULL</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">reverse_lines</span>(input, output);</span><br><span class="line"> <span class="built_in">fclose</span>(input);</span><br><span class="line"> <span class="built_in">fclose</span>(output);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> <span class="keyword">default</span>: <span class="comment">// 参数过多 test 1</span></span><br><span class="line"> <span class="built_in">fprintf</span>(stderr, <span class="string">"usage: reverse <input> <output>\n"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 从 'input' 读取行并逆序写入 'output'</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">reverse_lines</span><span class="params">(FILE *input, FILE *output)</span> </span>{</span><br><span class="line"> <span class="type">char</span> *line = <span class="literal">NULL</span>;</span><br><span class="line"> <span class="type">char</span> **lines = <span class="literal">NULL</span>;</span><br><span class="line"> <span class="type">size_t</span> len = <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> num_lines = <span class="number">0</span>;</span><br><span class="line"> <span class="type">int</span> capacity = <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> lines = <span class="built_in">malloc</span>(capacity * <span class="built_in">sizeof</span>(<span class="type">char</span> *));</span><br><span class="line"> <span class="comment">// 读取行到动态扩展数组中</span></span><br><span class="line"> <span class="keyword">while</span> ((<span class="built_in">getline</span>(&line, &len, input)) != <span class="number">-1</span>) { </span><br><span class="line"> <span class="comment">// 将input一行放入缓冲区中，getline会自动分配缓冲区line的长度len</span></span><br><span class="line"> <span class="keyword">if</span> (num_lines >= capacity) {</span><br><span class="line"> capacity *= <span class="number">2</span>;</span><br><span class="line"> <span class="comment">// 重新分配内存</span></span><br><span class="line"> lines = <span class="built_in">realloc</span>(lines, capacity * <span class="built_in">sizeof</span>(<span class="type">char</span> *));</span><br><span class="line"> }</span><br><span class="line"> lines[num_lines++] = <span class="built_in">strdup</span>(line); <span class="comment">// 根据缓冲区内的数据创建新的字符串</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 逆序输出行</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = num_lines - <span class="number">1</span>; i >= <span class="number">0</span>; i--) {</span><br><span class="line"> <span class="built_in">fprintf</span>(output, <span class="string">"%s"</span>, lines[i]);</span><br><span class="line"> <span class="built_in">free</span>(lines[i]);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">free</span>(lines);</span><br><span class="line"> <span class="built_in">free</span>(line);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 打印错误消息到 stderr 并以代码1退出</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">print_error_and_exit</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *message, <span class="type">const</span> <span class="type">char</span> *file)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (file) {</span><br><span class="line"> <span class="built_in">fprintf</span>(stderr, <span class="string">"%s '%s'\n"</span>, message, file);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">fprintf</span>(stderr, <span class="string">"%s\n"</span>, message);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="运行结果"><a href="#运行结果" class="headerlink" title="运行结果"></a>运行结果</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113244927.png" alt=""></p><h3 id="实验2——seucat程序实现"><a href="#实验2——seucat程序实现" class="headerlink" title="实验2——seucat程序实现"></a>实验2——seucat程序实现</h3><p>要求：通过命令行调用一个简单的seucat程序（C语言）</p><h4 id="设计思路-1"><a href="#设计思路-1" class="headerlink" title="设计思路"></a>设计思路</h4><ol><li><p>编写程序</p><ul><li>编写一个 C 程序 <code>seucat.c</code>，读取用户指定的文件并打印其内容。</li></ul></li><li><p>编译程序</p><ul><li><p>使用 GCC 编译 <code>seucat.c</code>，生成可执行文件 seucat。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc seucat.c -o seucat</span><br></pre></td></tr></table></figure></li></ul></li><li><p>测试程序</p><ul><li><p>使用提供的测试脚本对编译的可执行文件进行测试。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./test-seucat.sh</span><br></pre></td></tr></table></figure></li></ul></li></ol><h4 id="代码流程图-1"><a href="#代码流程图-1" class="headerlink" title="代码流程图"></a>代码流程图</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113253579.jpeg" alt="画板"></p><h4 id="源代码-1"><a href="#源代码-1" class="headerlink" title="源代码"></a>源代码</h4><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">print_file_content</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *filename)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> </span>{</span><br><span class="line"> <span class="comment">// 如果没有指定文件，正常退出并返回状态码0</span></span><br><span class="line"> <span class="keyword">if</span> (argc == <span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 遍历命令行参数中的每个文件名</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">1</span>; i < argc; i++) {</span><br><span class="line"> <span class="built_in">print_file_content</span>(argv[i]);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 打印文件内容</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">print_file_content</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *filename)</span> </span>{</span><br><span class="line"> FILE *file = <span class="built_in">fopen</span>(filename, <span class="string">"r"</span>);</span><br><span class="line"> <span class="keyword">if</span> (file == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"cannot open file\n"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">char</span> buffer[<span class="number">1024</span>];</span><br><span class="line"> <span class="comment">// 使用 fgets 按行读取并打印文件内容</span></span><br><span class="line"> <span class="keyword">while</span> (<span class="built_in">fgets</span>(buffer, <span class="built_in">sizeof</span>(buffer), file)) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"%s"</span>, buffer);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">fclose</span>(file);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="运行结果-1"><a href="#运行结果-1" class="headerlink" title="运行结果"></a>运行结果</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113306405.png" alt=""></p><h3 id="实验3——seugrep程序实现"><a href="#实验3——seugrep程序实现" class="headerlink" title="实验3——seugrep程序实现"></a>实验3——seugrep程序实现</h3><p>要求：通过命令行调用一个简单的seugrep程序（C语言）</p><h4 id="设计思路-2"><a href="#设计思路-2" class="headerlink" title="设计思路"></a>设计思路</h4><pre><code>1. 编写程序+ 编写一个 C 程序 `seugrep.c`，逐行查看一个文件，并在该行内找到用户指定的搜索词。如果存在，则打印该行。</code></pre><ol><li><p>编译程序</p><ul><li><p>使用 GCC 编译 <code>seugrep.c</code>，生成可执行文件 seugrep。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc seugrep.c -o seugrep</span><br></pre></td></tr></table></figure></li></ul></li><li><p>测试程序</p><ul><li><p>使用提供的测试脚本对编译的可执行文件进行测试。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./test-seugrep.sh</span><br></pre></td></tr></table></figure></li></ul></li></ol><h4 id="代码流程图-2"><a href="#代码流程图-2" class="headerlink" title="代码流程图"></a>代码流程图</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113311386.jpeg" alt="画板"></p><h4 id="源代码-2"><a href="#源代码-2" class="headerlink" title="源代码"></a>源代码</h4><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><string.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">search_in_file</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *search_term, FILE *file)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (argc == <span class="number">1</span>) {</span><br><span class="line"> <span class="comment">// test 3</span></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"searchterm [file ...]\n"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="type">const</span> <span class="type">char</span> *search_term = argv[<span class="number">1</span>]; <span class="comment">// 搜索项</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果只提供了搜索项，则从标准输入读取数据</span></span><br><span class="line"> <span class="keyword">if</span> (argc == <span class="number">2</span>) {</span><br><span class="line"> <span class="comment">// test 4</span></span><br><span class="line"> <span class="built_in">search_in_file</span>(search_term, stdin);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 遍历命令行中指定的文件</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">2</span>; i < argc; i++) {</span><br><span class="line"> FILE *file = <span class="built_in">fopen</span>(argv[i], <span class="string">"r"</span>);</span><br><span class="line"> <span class="keyword">if</span> (file == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">// test 2</span></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"cannot open file\n"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">search_in_file</span>(search_term, file); </span><br><span class="line"> <span class="built_in">fclose</span>(file); </span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 在指定文件中搜索字符串</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">search_in_file</span><span class="params">(<span class="type">const</span> <span class="type">char</span> *search_term, FILE *file)</span> </span>{</span><br><span class="line"> <span class="type">char</span> line[<span class="number">1024</span>];</span><br><span class="line"> <span class="keyword">while</span> (<span class="built_in">fgets</span>(line, <span class="built_in">sizeof</span>(line), file)) { </span><br><span class="line"> <span class="keyword">if</span> (<span class="built_in">strstr</span>(line, search_term)) { </span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"%s"</span>, line); </span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="运行结果-2"><a href="#运行结果-2" class="headerlink" title="运行结果"></a>运行结果</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113319415.png" alt=""></p><h3 id="实验4——seuzip程序实现"><a href="#实验4——seuzip程序实现" class="headerlink" title="实验4——seuzip程序实现"></a>实验4——seuzip程序实现</h3><p>要求：通过命令行调用一个简单的seuzip程序（C语言）</p><h4 id="设计思路-3"><a href="#设计思路-3" class="headerlink" title="设计思路"></a>设计思路</h4><ol><li><p>编写程序</p><ul><li>编写一个 C 程序 <code>seuzip.c</code>，压缩文件工具。这里的压缩方式使用RLE压缩算法：当在一行中遇到n个相同类型的字符时，seuzip 将其变成数字n和一个该字符示例。</li></ul></li><li><p>编译程序</p><ul><li><p>使用 GCC 编译 <code>seuzip.c</code>，生成可执行文件 seuzip。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc seuzip.c -o seuzip</span><br></pre></td></tr></table></figure></li></ul></li><li><p>测试程序</p><ul><li><p>使用提供的测试脚本对编译的可执行文件进行测试。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./test-seuzip.sh</span><br></pre></td></tr></table></figure></li></ul></li></ol><h4 id="代码流程图-3"><a href="#代码流程图-3" class="headerlink" title="代码流程图"></a>代码流程图</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113325462.jpeg" alt="画板"></p><h4 id="源代码-3"><a href="#源代码-3" class="headerlink" title="源代码"></a>源代码</h4><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">zip_file</span><span class="params">(FILE *input, FILE *output)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (argc < <span class="number">2</span>) {</span><br><span class="line"> <span class="comment">// test 3</span></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"seuzip: file1 [file2 ...]\n"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 汇总所有输入文件的内容 test 2</span></span><br><span class="line"> FILE *temp_file = <span class="built_in">tmpfile</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 将所有输入文件内容写入临时文件</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">1</span>; i < argc; i++) {</span><br><span class="line"> FILE *file = <span class="built_in">fopen</span>(argv[i], <span class="string">"r"</span>);</span><br><span class="line"></span><br><span class="line"> <span class="type">char</span> buffer[<span class="number">1024</span>];</span><br><span class="line"> <span class="type">size_t</span> bytes_read;</span><br><span class="line"> <span class="keyword">while</span> ((bytes_read = <span class="built_in">fread</span>(buffer, <span class="number">1</span>, <span class="built_in">sizeof</span>(buffer), file)) > <span class="number">0</span>) {</span><br><span class="line"> <span class="built_in">fwrite</span>(buffer, <span class="number">1</span>, bytes_read, temp_file);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">fclose</span>(file);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 回到临时文件的开头</span></span><br><span class="line"> <span class="built_in">rewind</span>(temp_file);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 压缩临时文件内容并输出到标准输出</span></span><br><span class="line"> <span class="built_in">zip_file</span>(temp_file, stdout);</span><br><span class="line"></span><br><span class="line"> <span class="built_in">fclose</span>(temp_file);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 压缩文件内容</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">zip_file</span><span class="params">(FILE *input, FILE *output)</span> </span>{</span><br><span class="line"> <span class="type">int</span> count = <span class="number">0</span>;</span><br><span class="line"> <span class="type">char</span> current, previous;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> ((previous = <span class="built_in">fgetc</span>(input)) == EOF) {</span><br><span class="line"> <span class="keyword">return</span>; <span class="comment">// 空文件</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> count = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">while</span> ((current = <span class="built_in">fgetc</span>(input)) != EOF) {</span><br><span class="line"> <span class="keyword">if</span> (current == previous) {</span><br><span class="line"> count++;</span><br><span class="line"> } </span><br><span class="line"> <span class="keyword">else</span> {</span><br><span class="line"> <span class="built_in">fwrite</span>(&count, <span class="built_in">sizeof</span>(<span class="type">int</span>), <span class="number">1</span>, output); <span class="comment">// 写入重复计数</span></span><br><span class="line"> <span class="built_in">fwrite</span>(&previous, <span class="built_in">sizeof</span>(<span class="type">char</span>), <span class="number">1</span>, output); <span class="comment">// 写入字符</span></span><br><span class="line"> previous = current;</span><br><span class="line"> count = <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 写入最后一组字符(current == previous is true)</span></span><br><span class="line"> <span class="built_in">fwrite</span>(&count, <span class="built_in">sizeof</span>(<span class="type">int</span>), <span class="number">1</span>, output);</span><br><span class="line"> <span class="built_in">fwrite</span>(&previous, <span class="built_in">sizeof</span>(<span class="type">char</span>), <span class="number">1</span>, output);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="运行结果-3"><a href="#运行结果-3" class="headerlink" title="运行结果"></a>运行结果</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113332471.png" alt=""></p><h3 id="实验5——seuunzip程序实现"><a href="#实验5——seuunzip程序实现" class="headerlink" title="实验5——seuunzip程序实现"></a>实验5——seuunzip程序实现</h3><p>要求：通过命令行调用一个简单的seuunzip程序（C语言）</p><h4 id="设计思路-4"><a href="#设计思路-4" class="headerlink" title="设计思路"></a>设计思路</h4><ol><li><p>编写程序</p><ul><li>编写一个 C 程序 <code>seuunzip.c</code>，与seuzip工具相反，接收压缩文件并写入未压缩结果。</li></ul></li><li><p>编译程序</p><ul><li><p>使用 GCC 编译 <code>seuunzip.c</code>，生成可执行文件 seuunzip。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc seuunzip.c -o seuunzip</span><br></pre></td></tr></table></figure></li></ul></li><li><p>测试程序</p><ul><li><p>使用提供的测试脚本对编译的可执行文件进行测试。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./test-seuunzip.sh</span><br></pre></td></tr></table></figure></li></ul></li></ol><h4 id="代码流程图-4"><a href="#代码流程图-4" class="headerlink" title="代码流程图"></a>代码流程图</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113339115.jpeg" alt="画板"></p><h4 id="源代码-4"><a href="#源代码-4" class="headerlink" title="源代码"></a>源代码</h4><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdio.h></span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string"><stdlib.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">unzip_file</span><span class="params">(FILE *input, FILE *output)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">(<span class="type">int</span> argc, <span class="type">char</span> *argv[])</span> </span>{</span><br><span class="line"> <span class="keyword">if</span> (argc < <span class="number">2</span>) {</span><br><span class="line"> <span class="comment">// test 3</span></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"seuunzip: file1 [file2 ...]\n"</span>);</span><br><span class="line"> <span class="built_in">exit</span>(<span class="number">1</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 对每个输入文件进行解压</span></span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">1</span>; i < argc; i++) {</span><br><span class="line"> <span class="comment">// test 2</span></span><br><span class="line"> FILE *file = <span class="built_in">fopen</span>(argv[i], <span class="string">"rb"</span>); </span><br><span class="line"> <span class="built_in">unzip_file</span>(file, stdout);</span><br><span class="line"> <span class="built_in">fclose</span>(file);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">unzip_file</span><span class="params">(FILE *input, FILE *output)</span> </span>{</span><br><span class="line"> <span class="type">int</span> count;</span><br><span class="line"> <span class="type">char</span> character;</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 逐组读取压缩数据</span></span><br><span class="line"> <span class="keyword">while</span> (<span class="built_in">fread</span>(&count, <span class="built_in">sizeof</span>(<span class="type">int</span>), <span class="number">1</span>, input) == <span class="number">1</span>) { </span><br><span class="line"> <span class="built_in">fread</span>(&character, <span class="built_in">sizeof</span>(<span class="type">char</span>), <span class="number">1</span>, input);</span><br><span class="line"> <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">0</span>; i < count; i++) {</span><br><span class="line"> <span class="built_in">fputc</span>(character, output);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="运行结果-4"><a href="#运行结果-4" class="headerlink" title="运行结果"></a>运行结果</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124113345740.png" alt=""></p><h3 id="实验体会"><a href="#实验体会" class="headerlink" title="实验体会"></a>实验体会</h3><ul><li>Linux环境搭建的选择<ul><li>初始我选择使用VirtualBox虚拟机来安装Fedora 7.0 作为我的Linux开发环境，第一次打开Fedora时会自动安装gcc等开发环境，这对完成本次实验来说是一个便利。但在后续使用时，由于虚拟机的性能问题和一些窗口分辨率，键鼠的bug，导致我使用的时候感到卡顿和不适。并且Fedora的图形化界面其实在本次实验中是没有必要的，图形化界面反而使得进行简单操作的步骤增加。因此最终我选择在WSL2中安装Ubuntu作为Linux的开发环境。</li><li>WSL (Windows Subsystem for Linux) 是由 Microsoft 开发的一种在 Windows 操作系统中运行 Linux 环境的兼容层。它允许用户直接在 Windows 上运行 Linux 命令行工具、应用程序和开发环境，而无需额外安装虚拟机或双系统。WSL的性能损耗小，并且搭配vscode的开发体验十分便利，可以便捷地使用vscode的文件管理系统和终端系统。</li></ul></li><li><p>编译程序中的小插曲</p><ul><li><p>在编译 .c 文件时，一开始我使用了如下指令</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc -c reverse.c -o reverse</span><br></pre></td></tr></table></figure><ul><li>因此在运行测试脚本时无法通过。原因在于上面的指令使用了<code>-c</code> 参数，仅编译代码生成目标文件（object file），不会链接生成可执行文件。输出文件是机器可读的二进制目标文件，通常带 <code>.o</code> 后缀。这种方法通常用在多文件项目中，每个源文件单独编译生成目标文件，最后将所有目标文件链接成一个可执行文件。</li><li>将编译指令换回如下后就正确了。</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gcc -o reverse reverse.c</span><br></pre></td></tr></table></figure></li></ul></li></ul><div class="table-container"><table><thead><tr><th><strong>命令</strong></th><th><strong>特点</strong></th><th><strong>输出文件</strong></th><th><strong>用途</strong></th></tr></thead><tbody><tr><td><code>gcc -c reverse.c -o reverse</code></td><td>只编译，不链接，生成目标文件（非可执行文件）。</td><td><code>reverse.o</code>（目标文件）</td><td>多文件编译项目的中间步骤。</td></tr><tr><td><code>gcc -o reverse reverse.c</code></td><td>编译并链接，生成完整的可执行文件。</td><td><code>reverse</code>（可执行文件）</td><td>用于生成单文件的可执行程序。</td></tr></tbody></table></div><ul><li>本次实验部分应该是参考了MIT的Xv6操作系统实验，这些实验内容提供了标准样例来检测程序的正确性。这些示例程序有些是为了检验对文件打开的异常处理，有些是为了检验对命令行参数错误的异常处理，检测样例固定了输入输出，因此在实验过程中很容易因为输出的格式错误导致不通过。在面对这样的检验方法时，最好的方法是查看其提供的标准输入输出并且一边尝试一边修改代码。<ul><li>例如在实现seuzip时，当命令行参数给的是多个文件时，是分文件压缩还是将所有文件中的内容整合起来再压缩？当没有看明白实验要求的时候，查看示例输入输出就能很好的解决这个问题。</li></ul></li></ul>]]></content>
<categories>
<category> 学习笔记 </category>
</categories>
<tags>
<tag> OS </tag>
<tag> 操作系统 </tag>
<tag> 实验报告 </tag>
</tags>
</entry>
<entry>
<title>【ML-BTC复现】多标记学习领域算法复现（多标签二叉分类树）</title>
<link href="/2025/01/17/%E3%80%90ML-BTC%E5%A4%8D%E7%8E%B0%E3%80%91%E5%A4%9A%E6%A0%87%E8%AE%B0%E5%AD%A6%E4%B9%A0%E9%A2%86%E5%9F%9F%E7%AE%97%E6%B3%95%E5%A4%8D%E7%8E%B0%EF%BC%88%E5%A4%9A%E6%A0%87%E7%AD%BE%E4%BA%8C%E5%8F%89%E5%88%86%E7%B1%BB%E6%A0%91%EF%BC%89/"/>
<url>/2025/01/17/%E3%80%90ML-BTC%E5%A4%8D%E7%8E%B0%E3%80%91%E5%A4%9A%E6%A0%87%E8%AE%B0%E5%AD%A6%E4%B9%A0%E9%A2%86%E5%9F%9F%E7%AE%97%E6%B3%95%E5%A4%8D%E7%8E%B0%EF%BC%88%E5%A4%9A%E6%A0%87%E7%AD%BE%E4%BA%8C%E5%8F%89%E5%88%86%E7%B1%BB%E6%A0%91%EF%BC%89/</url>
<content type="html"><![CDATA[<p>复现代码在<a href="https://github.com/Warma10032/ML-BTC">https://github.com/Warma10032/ML-BTC</a></p><h2 id="任务简介"><a href="#任务简介" class="headerlink" title="任务简介"></a>任务简介</h2><ol><li><p>领域背景</p><p>多标签分类（multi-label classification ）指的是一个输入的样本可以同时拥有几个类别标签，比如一首歌的标签可以是流行、轻快，一部电影的标签可以是动作、喜剧、搞笑，一本书的标签可以是经典、文学等，这都是多标签分类的情况。多标签分类的一个重要特点是样本的所有标签是不具有排他性的。</p><p>处理多标签数据时，复杂的决策空间是面临的最主要的问题。与单标签分类不同，多标签分类需学习更复杂决策边界，现有多标签分类技术存在许多不足。</p></li><li><p>典型算法</p><ol><li><p>复杂单分类器</p><p>通过一个复杂的单一分类器来处理多标签分类任务。分类器需要学习所有标签的决策边界。</p><ul><li>优点：可以一次性考虑所有标签的关系，适用于特定规模的数据。</li><li>缺点：训练复杂且耗时，尤其在标签数量或数据量很大的情况下。对模型复杂度提出要求。</li></ul></li><li><p>集成分类器</p><p>使用多个分类器的组合来处理多标签问题。</p><ul><li>优点：灵活，可扩展，易于实现。可以利用现有的单标签分类器。</li><li>缺点：往往忽略了标签间的相关性（如集成一对多分类器），或需要显式编码标签间关系（如分类器链），增加了复杂度。</li></ul></li></ol></li><li><p>论文的算法</p><p>论文中提出的<strong>多标签二叉树分类器</strong><a href="#reference1">[1]</a>，是一个基于决策树的分层多标签分类器，利用标签相关性，将数据按类标签分割，用于训练多个小分类器。ML-BTC采取的是一种层次聚类的方法，它将训练集根据标签集的汉明距离进行层次二分聚类，从而将大数据集分为多个小的分类任务；同时考虑了标签相关性。根据每次分类的数据特点，模型会选择对应的分裂策略和分类器。用合适的分类器，解决不同问题；可以有效缓解数据分布不平衡的影响，同时早停策略可以避免过拟合</p><p>ML-BTC 的设计目标是通过一种层次化、结构化的方法来解决标签相关性、类别不平衡、决策空间复杂性等多标签分类中的核心挑战，同时减少训练和推理的计算成本。这种方法在多种规模的数据集上都表现出了较好的分类效果，尤其在处理类别不平衡和高维数据方面具有显著优势。</p></li></ol><h2 id="算法介绍"><a href="#算法介绍" class="headerlink" title="算法介绍"></a><strong>算法介绍</strong></h2><p><strong>多标签二叉树分类器</strong>（Multi-Label Binary Tree Classifier, MLBTC）是一种基于树结构的多标签分类算法。该算法通过递归地将标签空间划分为两个子集，构建一个二叉树结构来处理多标签分类问题。在每个内部节点，算法使用基于汉明距离的二分聚类——将当前节点的标签集分为两个子集，选择汉明距离最大的两个标签作为聚类中心点，其余标签根据到这两个中心点的汉明距离进行分配。</p><p>在训练过程中，对于每个内部节点，算法会训练一个二元分类器来预测样本应该被分到左子树还是右子树。这种分层的方式不仅考虑了标签之间的相关性，还通过树结构降低了问题的复杂度。对于每个叶子节点，会根据叶子节点中的标签树，选择是否再训练一个分类器和选择训练何种分类器。</p><p>在预测阶段，测试样本从根节点开始，根据每个内部节点的分类器决策结果，逐步向下传递直到达到叶节点，在叶子节点中获得完整的标签预测结果。</p><p>该算法的主要优势在于：</p><ol><li>通过树结构将复杂的多标签分类问题分解为一系列更简单的二分类问题</li><li>考虑了标签之间的相关性，特别是通过汉明距离来度量标签的相似度，用标签的相似度来进行分裂。</li><li>模型会选择对应的分裂策略和分类器。用合适的分类器，可以有效缓解数据分布不平衡的影响。</li><li>模型的早停策略不强求分裂叶子节点直到其中只剩一个类别，这种方法避免模型过于复杂，减轻了过拟合风险，降低了计算量。</li></ol><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124190059259.png" alt="训练过程流程图"></p><p><em>其中的H，C分别指的是节点中的多标签熵和样本基数，</em></p><p><em>分别表示的是节点中样本标签的多样性和样本个数。</em></p><p><strong>具体解释训练流程：</strong></p><ol><li><p>H>H阈值 且 C>C阈值</p><p>子集之间大小不平衡：使用 k-NN 分类器处理此节点。</p><p>子集之间大小相对平衡：使用 SVM 分类器处理此节点。</p></li><li><p>H<H阈值 且 C>C阈值</p><p>在该节点训练一个多层感知机（MLP）分类器</p><p>MLP 适合处理相对集中的样本，能够有效学习不同类别之间的决策边界。</p></li><li><p>C<C阈值</p><p>在该节点使用 ML-kNN 分类器</p><p>ML-kNN 能够在小样本条件下有效分类，并保留多标签的决策能力。</p></li><li><p>H=0</p><p>表示该节点中仅有一个标签，将它分配给抵达的节点即可。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250124190124201.png" alt="推理过程流程图"></p></li></ol><p><strong>具体解释推理流程：</strong></p><ol><li><p>根节点</p><p>未知样本从根节点出发，使用根节点中训练好的二分类器被分类到其中一个子节点中。</p></li><li><p>中间节点</p><p>每个中间节点包含一个二分类器，样本经过每个中间节点会被分配到左或右子节点。</p></li><li><p>叶子节点</p><p>根据叶子节点中的标签类别或分类器，确定最终的类别。</p></li></ol><h2 id="算法复现"><a href="#算法复现" class="headerlink" title="算法复现"></a>算法复现</h2><h3 id="数据集"><a href="#数据集" class="headerlink" title="数据集"></a>数据集</h3><p>本次实验的数据集从<a href="https://www.uco.es/kdis/mllresources/">Multi-Label Classification Dataset Repository</a> 下载。</p><p>数据集中所有属性都已经进行了数值化，所以在这个网址下载的数据集存储大小都很小。</p><p>我们选取了论文中相同的几个数据集进行复现，按论文中的大小分类，small，medium，large三类都进行了复现。</p><div class="table-container"><table><thead><tr><th>Dataset</th><th>数据类型</th><th>样本条数</th><th>属性个数</th><th>标签维度</th></tr></thead><tbody><tr><td>Cal500</td><td>Music</td><td>502</td><td>68</td><td>174</td></tr><tr><td>Emotions</td><td>Music</td><td>593</td><td>72</td><td>6</td></tr><tr><td>Flags</td><td>Image</td><td>194</td><td>19</td><td>7</td></tr><tr><td>Enron</td><td>Text</td><td>1702</td><td>1001</td><td>53</td></tr><tr><td>Yeast</td><td>Biology</td><td>2417</td><td>103</td><td>14</td></tr><tr><td>Bibtex</td><td>Text</td><td>7395</td><td>1836</td><td>159</td></tr><tr><td>Delicious</td><td>Text</td><td>16110</td><td>500</td><td>983</td></tr><tr><td>Yelp</td><td>Text</td><td>10810</td><td>671</td><td>5</td></tr></tbody></table></div><h3 id="实验设置"><a href="#实验设置" class="headerlink" title="实验设置"></a>实验设置</h3><ol><li><p>实验评估设置</p><p>实验评估策略：5折交叉验证</p><p>实验评估指标：</p><ol><li><p>Hamming Loss (HL)</p><p>预测标记集与真实标记集的不匹配率</p><p>计算每个样本预测标记集与真实标记集的对称差集大小，再除以标记集总数，最后取平均</p></li><li><p>Subset Accuracy (SA)</p><p>完全匹配的准确率</p><p>只有预测标记集与真实标记集完全相同才算正确</p></li><li><p>Macro-F1 (MacF1)</p><p>先对每个标记计算F1值，然后取平均，对所有类别的处理权重相同</p><p>不考虑样本分布不均衡</p></li><li><p>Micro-F1 (MicF1)</p><p>将所有标记的预测结果混在一起计算整体的F1值</p><p>考虑了样本分布，更关注高频标记的表现</p></li><li><p>F-Measure (FM)</p><p>基于样本级别的F1度量</p><p>对每个样本计算预测标记集与真实标记集的F1值，然后取平均</p></li><li><p>Accuracy (Acc)</p><p>预测正确的标记占总标记数的比例</p><p>计算预测标记集与真实标记集的交集大小除以标记集总数</p></li><li><p>Geometric Mean (GM)</p><p>计算每个标记正类和负类的召回率</p><p>计算两个召回率的几何平均</p><p>这种GM计算方式特别关注模型在正类和负类上的平衡性能，能够有效处理类别不平衡问题</p></li></ol></li><li><p>模型与参数设置</p><p>H和C的阈值设置：</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">self.h_threshold = self.<span class="built_in">compute_ml_entropy</span>(y) / self.h_threshold_den</span><br><span class="line">self.c_threshold = <span class="built_in">len</span>(X) / self.c_threshold_den</span><br></pre></td></tr></table></figure><p>在实际代码中，我们使用的是根节点（初始训练集）的多标签熵和样本总数的1/10来作为叶子节点不进行分裂的阈值。</p></li></ol><h3 id="复现代码说明"><a href="#复现代码说明" class="headerlink" title="复现代码说明"></a>复现代码说明</h3><ol><li><p>树是怎么递归构建的（build_tree函数）</p><p>通过对节点判断是否进行分裂，如果进行分裂，执行二分数据集代码，并选择合适的分类器。</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br></pre></td><td class="code"><pre><span class="line"><span class="function">def <span class="title">build_tree</span><span class="params">(self, X, y)</span>:</span></span><br><span class="line"><span class="function"> <span class="string">""</span><span class="string">"构建分类器树"</span><span class="string">""</span></span></span><br><span class="line"><span class="function"> node =</span> <span class="built_in">MLBTCNode</span>()</span><br><span class="line"></span><br><span class="line"> # 检查停止条件</span><br><span class="line"> current_entropy = self.<span class="built_in">compute_ml_entropy</span>(y)</span><br><span class="line"> <span class="keyword">if</span> current_entropy < self.h_threshold <span class="keyword">or</span> <span class="built_in">len</span>(X) < self.c_threshold:</span><br><span class="line"> node.is_leaf = True</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(np.<span class="built_in">unique</span>(y, axis=<span class="number">0</span>)) == <span class="number">1</span>:</span><br><span class="line"> node.label_set = y[<span class="number">0</span>]</span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> node.ml_classifier = <span class="built_in">MLPClassifier</span>(</span><br><span class="line"> hidden_layer_sizes=(<span class="number">100</span>,), max_iter=<span class="number">500</span></span><br><span class="line"> )</span><br><span class="line"> node.ml_classifier.<span class="built_in">fit</span>(X, y)</span><br><span class="line"> <span class="keyword">return</span> node</span><br><span class="line"></span><br><span class="line"> # 划分数据</span><br><span class="line"> clusters = self.<span class="built_in">split_data</span>(X, y)</span><br><span class="line"></span><br><span class="line"> # 如果无法划分，创建叶节点</span><br><span class="line"> <span class="keyword">if</span> <span class="built_in">len</span>(np.<span class="built_in">unique</span>(clusters)) == <span class="number">1</span>:</span><br><span class="line"> node.is_leaf = True</span><br><span class="line"> node.ml_classifier = <span class="built_in">MLPClassifier</span>(hidden_layer_sizes=(<span class="number">100</span>,), max_iter=<span class="number">500</span>)</span><br><span class="line"> node.ml_classifier.<span class="built_in">fit</span>(X, y)</span><br><span class="line"> <span class="keyword">return</span> node</span><br><span class="line"></span><br><span class="line"> # 选择并训练分类器</span><br><span class="line"> node.classifier = self.<span class="built_in">select_classifier</span>(X, clusters)</span><br><span class="line"> node.classifier.<span class="built_in">fit</span>(X, clusters)</span><br><span class="line"></span><br><span class="line"> # 递归构建子树</span><br><span class="line"> left_mask = clusters == <span class="number">0</span></span><br><span class="line"> right_mask = clusters == <span class="number">1</span></span><br><span class="line"></span><br><span class="line"> node.left = self.<span class="built_in">build_tree</span>(X[left_mask], y[left_mask])</span><br><span class="line"> node.right = self.<span class="built_in">build_tree</span>(X[right_mask], y[right_mask])</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> node</span><br></pre></td></tr></table></figure></li><li><p>树的二分裂中数据聚类算法是怎样实现的（split_data函数）</p><ol><li>计算所有标签对之间的汉明距离</li><li>找到汉明距离最大的两个标签作为中心点</li><li>其他标签根据到两个中心点的距离进行分配</li></ol><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br></pre></td><td class="code"><pre><span class="line"><span class="function">def <span class="title">hamming_distance</span><span class="params">(a, b)</span>:</span></span><br><span class="line"><span class="function"> <span class="string">""</span><span class="string">"计算两个标签向量之间的汉明距离"</span><span class="string">""</span></span></span><br><span class="line"><span class="function"> return np.sum(a !=</span> b)</span><br><span class="line"></span><br><span class="line"><span class="function">def <span class="title">split_data</span><span class="params">(self, X, y)</span>:</span></span><br><span class="line"><span class="function"> <span class="string">""</span><span class="string">"</span></span></span><br><span class="line"><span class="string"><span class="function"> 基于最大汉明距离的标签空间划分</span></span></span><br><span class="line"><span class="string"><span class="function"> Args:</span></span></span><br><span class="line"><span class="string"><span class="function"> X: 特征矩阵</span></span></span><br><span class="line"><span class="string"><span class="function"> y: 标签矩阵 (n_samples, n_labels)</span></span></span><br><span class="line"><span class="string"><span class="function"> Returns:</span></span></span><br><span class="line"><span class="string"><span class="function"> clusters: 划分结果，0和1表示两个子集</span></span></span><br><span class="line"><span class="string"><span class="function"> "</span><span class="string">""</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function"> if len(X) <=</span> <span class="number">1</span>:</span><br><span class="line"> <span class="keyword">return</span> np.<span class="built_in">array</span>([<span class="number">0</span>] * <span class="built_in">len</span>(X))</span><br><span class="line"></span><br><span class="line"> n_samples = <span class="built_in">len</span>(y)</span><br><span class="line"></span><br><span class="line"> # <span class="number">1.</span> 计算所有标签对之间的汉明距离</span><br><span class="line"> distances = np.<span class="built_in">zeros</span>((n_samples, n_samples))</span><br><span class="line"> <span class="keyword">for</span> i in <span class="built_in">range</span>(n_samples):</span><br><span class="line"> <span class="keyword">for</span> j in <span class="built_in">range</span>(i + <span class="number">1</span>, n_samples):</span><br><span class="line"> dist = <span class="built_in">hamming_distance</span>(y[i], y[j])</span><br><span class="line"> distances[i][j] = dist</span><br><span class="line"> distances[j][i] = dist</span><br><span class="line"></span><br><span class="line"> # <span class="number">2.</span> 找到距离最大的两个标签索引</span><br><span class="line"> max_dist = <span class="number">0</span></span><br><span class="line"> center1_idx = <span class="number">0</span></span><br><span class="line"> center2_idx = <span class="number">0</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> i in <span class="built_in">range</span>(n_samples):</span><br><span class="line"> <span class="keyword">for</span> j in <span class="built_in">range</span>(i + <span class="number">1</span>, n_samples):</span><br><span class="line"> <span class="keyword">if</span> distances[i][j] > max_dist:</span><br><span class="line"> max_dist = distances[i][j]</span><br><span class="line"> center1_idx = i</span><br><span class="line"> center2_idx = j</span><br><span class="line"></span><br><span class="line"> # <span class="number">3.</span> 将其他标签分配到最近的中心点</span><br><span class="line"> clusters = np.<span class="built_in">zeros</span>(n_samples, dtype=<span class="type">int</span>)</span><br><span class="line"> clusters[center2_idx] = <span class="number">1</span> # 第二个中心点标记为<span class="number">1</span></span><br><span class="line"></span><br><span class="line"> # 对每个样本，计算到两个中心点的距离，分配到较近的中心点</span><br><span class="line"> <span class="keyword">for</span> i in <span class="built_in">range</span>(n_samples):</span><br><span class="line"> <span class="keyword">if</span> i != center1_idx <span class="keyword">and</span> i != center2_idx: # 跳过中心点</span><br><span class="line"> dist_to_center1 = <span class="built_in">hamming_distance</span>(y[i], y[center1_idx])</span><br><span class="line"> dist_to_center2 = <span class="built_in">hamming_distance</span>(y[i], y[center2_idx])</span><br><span class="line"> clusters[i] = <span class="number">0</span> <span class="keyword">if</span> dist_to_center1 <= dist_to_center2 <span class="keyword">else</span> <span class="number">1</span></span><br><span class="line"></span><br><span class="line"> # 打印一些信息以便调试</span><br><span class="line"> <span class="built_in">print</span>(f<span class="string">"Selected centers: {center1_idx} and {center2_idx}"</span>)</span><br><span class="line"> <span class="built_in">print</span>(f<span class="string">"Maximum hamming distance between centers: {max_dist}"</span>)</span><br><span class="line"> <span class="built_in">print</span>(f<span class="string">"Cluster sizes: {np.sum(clusters == 0)} and {np.sum(clusters == 1)}"</span>)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> clusters</span><br></pre></td></tr></table></figure></li><li><p>如何选择分类器的（select_classifier函数）</p><p>根据聚类形成的两个子集决定，如果两个子集之间不平衡或者样本个数较少，就使用KNN，反之使用SVM。</p><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="function">def <span class="title">select_classifier</span><span class="params">(self, X, y)</span>:</span></span><br><span class="line"><span class="function"> <span class="string">""</span><span class="string">"根据数据特征选择合适的分类器"</span><span class="string">""</span></span></span><br><span class="line"><span class="function"> n_samples =</span> <span class="built_in">len</span>(X)</span><br><span class="line"> class_counts = np.<span class="built_in">sum</span>(y, axis=<span class="number">0</span>)</span><br><span class="line"></span><br><span class="line"> # 检查类别不平衡情况</span><br><span class="line"> imbalance_ratio = np.<span class="built_in">max</span>(class_counts) / np.<span class="built_in">min</span>(class_counts)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> imbalance_ratio > <span class="number">10</span> <span class="keyword">or</span> n_samples < <span class="number">50</span>:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">KNeighborsClassifier</span>(n_neighbors=<span class="number">1</span>)</span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> <span class="keyword">return</span> <span class="built_in">SVC</span>(kernel=<span class="string">"rbf"</span>, probability=True)</span><br></pre></td></tr></table></figure></li></ol><h2 id="复现结果与分析"><a href="#复现结果与分析" class="headerlink" title="复现结果与分析"></a>复现结果与分析</h2><h3 id="复现结果"><a href="#复现结果" class="headerlink" title="复现结果"></a>复现结果</h3><p>使用我们复现的结果与论文在同数据集上进行对比，以下是对比的结果：</p><ul><li>Cal500数据集（small）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.1931</td><td>0.2052</td><td>0.1343</td><td>0.2458</td><td>0.3639</td><td>0.2241</td><td>0.5089</td></tr><tr><td>复现</td><td>0.1740</td><td>0.0000</td><td>0.1383</td><td>0.3533</td><td>0.3496</td><td>0.8260</td><td>0.1968</td></tr></tbody></table></div><ul><li>Emotions数据集（small）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.2126</td><td>0.6384</td><td>0.6352</td><td>0.6601</td><td>0.6704</td><td>0.5518</td><td>0.7018</td></tr><tr><td>复现</td><td>0.2029</td><td>0.3322</td><td>0.6453</td><td>0.6727</td><td>0.6455</td><td>0.7971</td><td>0.7258</td></tr></tbody></table></div><ul><li>Flags数据集（small）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.3438</td><td>0.5922</td><td>0.5085</td><td>0.6114</td><td>0.6697</td><td>0.5194</td><td>0.6115</td></tr><tr><td>复现</td><td>0.3197</td><td>0.1441</td><td>0.5770</td><td>0.6661</td><td>0.6396</td><td>0.6803</td><td>0.5232</td></tr></tbody></table></div><ul><li>Enron数据集（medium）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.0889</td><td>0.3622</td><td>0.1679</td><td>0.3724</td><td>0.4557</td><td>0.3119</td><td>0.5193</td></tr><tr><td>复现</td><td>0.0621</td><td>0.3854</td><td>0.6090</td><td>0.6082</td><td>0.2159</td><td>0.9379</td><td>0.7811</td></tr></tbody></table></div><ul><li>Yeast数据集（medium）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.2342</td><td>0.5275</td><td>0.3443</td><td>0.5605</td><td>0.6339</td><td>0.4319</td><td>0.6425</td></tr><tr><td>复现</td><td>0.2098</td><td>0.2362</td><td>0.4214</td><td>0.6482</td><td>0.6314</td><td>0.7902</td><td>0.4148</td></tr></tbody></table></div><ul><li>Bibtex数据集（large）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.0299</td><td>0.2919</td><td>0.1586</td><td>0.2597</td><td>0.3014</td><td>0.2391</td><td>0.3911</td></tr><tr><td>复现</td><td>0.0181</td><td>0.1596</td><td>0.1417</td><td>0.2778</td><td>0.3004</td><td>0.9819</td><td>0.2440</td></tr></tbody></table></div><ul><li>Delicious数据集（large）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.0381</td><td>0.2152</td><td>0.0824</td><td>0.2488</td><td>0.2439</td><td>0.1609</td><td>0.4012</td></tr><tr><td>复现</td><td>0.0242</td><td>0.0063</td><td>0.1243</td><td>0.2818</td><td>0.2738</td><td>0.9758</td><td>0.2277</td></tr></tbody></table></div><ul><li>Yelp数据集（large）：</li></ul><div class="table-container"><table><thead><tr><th></th><th>HL↓</th><th>SA↑</th><th>MacF1↑</th><th>MicF1↑</th><th>FM↑</th><th>Acc↑</th><th>GM↑</th></tr></thead><tbody><tr><td>论文</td><td>0.2167</td><td>0.6436</td><td>0.5944</td><td>0.6602</td><td>0.7104</td><td>0.5619</td><td>0.6872</td></tr><tr><td>复现</td><td>0.0901</td><td>0.6760</td><td>0.6802</td><td>0.7173</td><td>0.4859</td><td>0.9099</td><td>0.7470</td></tr></tbody></table></div><h3 id="结果分析"><a href="#结果分析" class="headerlink" title="结果分析"></a>结果分析</h3><p>在以上评估指标中论文和复现进行对比：</p><ul><li>可以看出HL和F1普遍比原论文更好。</li><li>在某些数据集的SA会明显低于原论文，例如Cal500，Delicious数据集，这些数据集都是典型的属性维度大于标签维度，如Cal500的属性维度为68，标签维度却有174，要实现SA即完全预测准确，我觉得这是很难的一件事，但是原始论文还是取得了0.2的得分，不清楚原始论文是否运用了额外的属性来进行预测，因为Cal500数据集其实是有多种属性（MFCC, MEL等），我们使用的是基础的属性。</li><li>Acc的验证可能有问题（按理说Acc应该和HL加和为1，不清楚原始论文是如何计算Acc的）。</li><li>GM较原论文波动较大。</li></ul><h2 id="参考文献"><a href="#参考文献" class="headerlink" title="参考文献"></a>参考文献</h2><p><code><a id="reference1"></code>[1]<code></a></code> A. Law and A. Ghosh, “Multi-Label Classification Using Binary Tree of Classifiers,” in IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 6, no. 3, pp. 677-689, June 2022, doi: 10.1109/TETCI.2021.3075717.</p>]]></content>
<categories>
<category> 科研日记 </category>
</categories>
<tags>
<tag> 人工智能 </tag>
<tag> 机器学习 </tag>
<tag> 多标记学习 </tag>
<tag> 复现 </tag>
</tags>
</entry>
<entry>
<title>OpenGL学习笔记</title>
<link href="/2024/10/04/OpenGL%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/"/>
<url>/2024/10/04/OpenGL%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/</url>
<content type="html"><![CDATA[<p>OpenGL学习和编码过程中的笔记总结，代码文件在：<a href="https://github.com/Warma10032/Computer-Graphics">https://github.com/Warma10032/Computer-Graphics</a></p><h1 id="实验内容"><a href="#实验内容" class="headerlink" title="实验内容"></a>实验内容</h1><ol><li>在第1次实验中，我们学习了OpenGL编程的基本规范，完成了一个简单的二维卡通图像。</li><li>在第2次实验中，我们进入三维的世界，因此我们重新搭建，通过导入模型和自己定义三维图形的顶点坐标，展示了一个鸡块和棒棒糖的组合画面，并让它们旋转平移动了起来。</li><li>在第3次实验中，我们学习了阴影和明暗，因此在第2次实验的基础上，我们添加了光源，通过光源和光照函数的实现，我们的模型上有了明暗变化。</li><li>在第4次实验中，我们给模型加上了纹理，通过在片元着色器中混合纹理颜色和光照颜色，实现了纹理和光照的结合，模型和背景也更加生动。</li><li>在最终的组合实验中，我们给画面加上了天空盒，并实现了场景漫游。用天空盒替换原来的黑色背景后明显更有真实感了，简单的第一人称场景漫游也给画面加上了一些交互要素。</li></ol><h1 id="设计思路和原理"><a href="#设计思路和原理" class="headerlink" title="设计思路和原理"></a>设计思路和原理</h1><p>下面是我认为在实验中的关键点的设计思路：</p><ol><li><p><strong>对物体运动的矩阵的设计</strong></p><p>基于计算机图形学中的坐标变换理论，需要将物体从局部坐标系转换到世界坐标系(M)，再转换到观察坐标系(V)，最后投影到屏幕(P)</p><p>模型矩阵(M)控制物体本身的变换：位置、旋转、缩放</p><p>视图矩阵(V)模拟相机移动，本质是将世界坐标系中的物体转换到相机坐标系</p><p>投影矩阵(P)将3D场景投影到2D平面，模拟人眼或相机的视觉效果</p><p>矩阵乘法顺序很重要：P<em>(V</em>M)，因为变换是从右向左应用的</p></li><li><p><strong>对光照模型的设计</strong></p><p>基于Phong光照模型，将光照分解为三个组成部分：</p><ul><li>环境光(Ambient)：模拟间接光照，使物体不会完全黑暗</li><li>漫反射光(Diffuse)：模拟光线照射到粗糙表面的散射效果</li><li>镜面反射光(Specular)：模拟光线在光滑表面的反射，产生高光</li></ul><p>通过设置以下四个参数就能为每个物体设置不同的光照材质：环境光反射系数；漫反射系数；镜面反射系数；光泽度。</p><p>在片元着色器中</p><ol><li><p>环境光计算：</p><p><code>全局环境光× 材质环境反射系数 + 光源环境光 × 材质环境反射系数</code></p><p>用于模拟来自所有方向的间接光照，不考虑位置和方向，确保物体在阴影处仍可见。</p></li><li><p>漫反射计算：</p><p>光源方向(L)和表面法向量(N)的点积决定漫反射强度，即用光源方向表面法向量的夹角来模拟现实世界的反射强度。</p></li><li><p>镜面反射计算：</p><p>反射方向(R)和视线方向(V)的点积决定高光强度，反射强度和上面相同，当反射方向射入摄像机便形成了高光，光泽度越大，高光区域越小越集中。</p></li></ol></li><li><p><strong>对法向量和曲面细分的设计</strong></p><p>从OpenGL光照实现原理可以看出，模型的法向量和曲面细分是十分重要的。</p><ol><li><p>法向量的物理意义：</p><p>表示物体表面的朝向，是光照计算的基础，用于计算光线入射角，决定漫反射强度；用于计算反射方向，决定镜面反射效果。</p></li><li><p>曲面细分的必要性：</p><p>增加多边形数量，使曲面更光滑；提供更精确的法向量，改善光照效果，使其更加真实，特别是对球体等曲面物体很重要。</p></li></ol></li><li><p><strong>对纹理映射系统的设计</strong></p><p>纹理坐标系统原理：</p><p>UV坐标范围在[0,1]之间，独立于物体实际大小，U坐标对应水平方向，V坐标对应垂直方向。通过为每个顶点指定纹理坐标，GPU通过插值计算片段的纹理坐标，根据纹理坐标从纹理图像采样颜色，我们便实现了纹理映射。</p></li><li><p>对纹理和明暗结合的设计</p><ol><li><p>基本原理：</p><p>纹理提供基础颜色信息，光照计算提供明暗变化，通过两者相乘得到最终效果。通过纹理采样获取物体表面的基础颜色信息，纹理颜色表示表面对不同波长光的反射率。再结合光照计算，光照颜色表示入射光的强度和颜色，将两者相乘得到最终反射光的颜色和强度。</p></li><li><p>同时我们还实现了渲染模式控制：</p><p>通过useLight开关控制渲染模式，对于天空盒等不需要光照计算的物体，我们只使用纹理颜色。</p></li></ol></li><li><p>对天空盒的设计</p><p>为什么使用天空盒？</p><p>目的是创造无限远的环境错觉，提供场景的背景上下文，增加场景的真实感。</p><p>使用立方体贴图，每个面对应一张图片，绘制时关闭深度测试让它永远绘制在其他物体之后，并移除天空盒视图矩阵的位移部分，使天空盒跟随相机移动。</p></li><li><p>对场景漫游系统设计</p><ol><li>相机位置和视角基于：位置(Position)——相机在世界空间的位置，方向(Front)——相机朝向的方向，上向量(Up)——定义相机的垂直方向，这三个lookat参数决定。</li><li>视角控制系统：使用水平角(Horizon)控制左右视角，垂直角(Vertical)控制上下视角，两个角度共同决定观察方向，使用三角函数计算实际的方向向量。</li><li>移动控制系统：通过键盘改变相机坐标，WASD控制前后左右移动，空格和Shift控制上下移动。</li></ol></li></ol><h1 id="OpenGL代码基本框架"><a href="#OpenGL代码基本框架" class="headerlink" title="OpenGL代码基本框架"></a>OpenGL代码基本框架</h1><p>在实验过过程中，老师提供的示例程序对我提供了很大的帮助，下面我想总结一下OpenGL代码基本框架，便于我后续使用能再次快速上手。</p><h2 id="OpenGL中的重要专有名词："><a href="#OpenGL中的重要专有名词：" class="headerlink" title="OpenGL中的重要专有名词："></a>OpenGL中的重要专有名词：</h2><ol><li><strong>缓冲区相关</strong><ul><li>VAO (Vertex Array Object)：顶点数组对象，存储顶点属性配置的容器，可以保存多个顶点属性的格式和对应VBO的引用。</li><li>VBO (Vertex Buffer Object)：顶点缓冲对象，在GPU内存中存储大量顶点数据的缓冲区，可以包含位置、颜色、纹理坐标等数据。</li><li>EBO/IBO (Element Buffer Object/Index Buffer Object)：索引缓冲对象，存储顶点索引的缓冲区，用于复用顶点数据，减少内存使用。</li></ul></li><li><strong>着色器相关</strong><ul><li>Shader（着色器）：在GPU上运行的小程序，用于处理图形渲染管线中的特定阶段。</li><li>Vertex Shader（顶点着色器）：处理单个顶点的着色器，负责顶点坐标变换和顶点属性计算。</li><li>Fragment Shader（片段着色器）：处理像素颜色的着色器，也称为像素着色器，决定每个像素的最终颜色。</li></ul></li><li><strong>矩阵变换相关</strong><ul><li>Model Matrix（模型矩阵）：将物体从局部坐标变换到世界坐标，包含平移、旋转、缩放操作。</li><li>View Matrix（视图矩阵）：将世界坐标变换到相机视角，定义观察者的位置和方向。</li><li>Projection Matrix（投影矩阵）：将3D场景投影到2D平面，包括透视投影和正交投影两种。</li><li>MVP Matrix：Model-View-Projection矩阵，三种矩阵的组合变换。</li></ul></li><li><strong>纹理相关</strong><ul><li>Texture（纹理）：用于给3D模型表面贴图的2D图像，可以存储颜色、法线等信息。</li><li>Texture Coordinates（纹理坐标）：也称UV坐标，定义纹理如何映射到3D模型表面。</li></ul></li><li><strong>渲染相关</strong><ul><li>Frame Buffer（帧缓冲区）：存储最终渲染图像的内存区域，包含颜色缓冲、深度缓冲等。</li><li>Depth Buffer（深度缓冲区）：存储每个像素的深度信息，用于处理3D物体的遮挡关系。</li></ul></li><li><strong>其他重要概念</strong><ul><li>Uniform变量：着色器程序中的全局变量，在渲染过程中保持不变。</li><li>Attribute（属性）：顶点的各种属性数据，如位置、颜色、法线等。</li><li>Primitive（图元）：基本绘制单位，包括点、线、三角形等。</li></ul></li></ol><h2 id="OpenGL开发的API框架"><a href="#OpenGL开发的API框架" class="headerlink" title="OpenGL开发的API框架"></a>OpenGL开发的API框架</h2><ol><li><strong>init函数的作用</strong><ul><li>初始化着色器程序并链接，加载纹理</li><li>设置相机初始位置和方向，设置物体初始位置，设置投影方式，初始化光照参数</li><li>设置顶点数据：包括创建VAO(顶点数组对象)和VBO(顶点缓冲对象)，指定顶点位置、纹理坐标、法线等数据，配置顶点属性指针，设置索引缓冲对象(EBO)</li></ul></li><li><strong>display函数的作用</strong><ul><li>缓冲区管理：包括清除颜色缓冲区和深度缓冲区，设置清除颜色，启用/禁用深度测试。</li><li>着色器程序使用：激活着色器程序，，获取uniform变量位置，传递uniform值到着色器。</li><li>矩阵变换：包括计算和更新模型矩阵(位移、旋转、缩放)，更新视图矩阵(相机位置和方向)，计算MVP矩阵，传递变换矩阵到着色器。</li><li>纹理绑定与使用：包括激活纹理单元，绑定纹理对象，设置纹理uniform采样器。</li></ul></li><li><strong>main函数的作用</strong><ul><li>GLFW窗口管理：包括初始化GLFW，创建窗口和OpenGL上下文，设置窗口属性和回调函数，处理窗口事件。</li><li>维护主渲染循环：包括处理输入事件，更新场景状态，调用渲染函数，交换缓冲区。</li><li>资源清理：包括释放OpenGL资源，清理GLFW资源，正确退出程序。</li></ul></li></ol><h1 id="碰到的问题总结"><a href="#碰到的问题总结" class="headerlink" title="碰到的问题总结"></a>碰到的问题总结</h1><ol><li>在实验中碰到的耗费了最多时间的问题是，在绘制自定义的三维物体时，总是出现绘制残缺的情况。一开始检查了很多地方都没检查出来问题，最后发现是在绘制时本应该使用 <code>glDrawArrays(GL_TRIANGLES, 0, myModel.getNumIndices())</code>，结果错误的使用了 <code>myModel.getNumVertices()</code>导致了错误，由于少绘制了顶点，从而导致模型残缺。</li><li>遇到的第二个问题是，在绘制阴影时，发现阴影效果不明显，检查了法向量的设置也没有发现什么问题。最终发现是因为没有绑定法向量的VBO导致的，修改后因为各面的法向量不同，使得阴影更有层次感了。</li><li>遇到的第三个问题就是，绘制天空盒时天空盒不显示，最终发现是由于深度测试的问题，由于天空盒应该显示在所有其他物体的最后面，我们可以在渲染时关闭深度测试，或者使用使用GL_LEQUAL深度测试函数来渲染最远端（最深）的物体。</li><li>最新问题：在最小化窗口时出现runtime error，发现原因是由于最小化，在计算视图比例aspect=height/width时出现了除以0的情况，增加了一个异常判断排除后，问题得以解决。</li></ol>]]></content>
<categories>
<category> 学习笔记 </category>
</categories>
<tags>
<tag> 计算机图形学 </tag>
<tag> OpenGL </tag>
</tags>
</entry>
<entry>
<title>视频异常检测领域论文综述</title>
<link href="/2024/05/10/%E8%A7%86%E9%A2%91%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B%E9%A2%86%E5%9F%9F%E8%AE%BA%E6%96%87%E7%BB%BC%E8%BF%B0/"/>
<url>/2024/05/10/%E8%A7%86%E9%A2%91%E5%BC%82%E5%B8%B8%E6%A3%80%E6%B5%8B%E9%A2%86%E5%9F%9F%E8%AE%BA%E6%96%87%E7%BB%BC%E8%BF%B0/</url>
<content type="html"><![CDATA[<h2 id="Abstract"><a href="#Abstract" class="headerlink" title="Abstract"></a>Abstract</h2><p>Video anomaly detection is an important research area in computer vision, with widespread applications in public safety, traffic management, healthcare, and other sectors. With the rapid development of event cameras, their high dynamic range, low latency, and low power consumption make them highly promising for anomaly detection tasks. This paper first introduces the working principles of event cameras and their advantages in the field of video anomaly detection. It then provides a detailed analysis of the main methods and approaches for video anomaly detection, including supervised and unsupervised learning, with the latter divided into reconstruction and prediction-based approaches. I also discuss the challenges faced in the current field, such as scene dependency, the diversity of abnormal behaviors, and data sparsity, and introduce recent advancements in research. Finally, I summarize the methods of modality fusion between event cameras and RGB cameras and propose future research directions and recommendations for improvement, aiming to provide insights for the further development of video anomaly detection tasks.</p><p><strong>Keywords</strong>: Computer Vision, Anomaly Detection, Event Cameras, Unsupervised Learning, Modality Fusion</p><h2 id="摘要"><a href="#摘要" class="headerlink" title="摘要"></a>摘要</h2><p>视频异常检测作为计算机视觉的一个重要研究方向，广泛应用于公共安全、交通管理、医疗健康等多个领域。随着事件相机的快速发展，其高动态范围、低延迟和低功耗等特点使其在异常检测任务中具有巨大的应用潜力。本文首先介绍了事件相机的工作原理及其在视频异常检测领域的优势。然后详细分析了现有视频异常检测的主要方法和思路，包括监督学习和无监督学习，在无监督学习中又分为重建和预测两大方向。接着，我探讨了当前领域面临的挑战，如场景依赖性、异常行为的多样性、数据稀疏性等，并介绍了近年来的前沿研究成果。最后，我对事件相机与RGB相机的模态融合方法进行了总结，并提出了未来的研究方向和改进建议，旨在为视频异常检测任务的进一步发展提供思路。</p><p><strong>关键词</strong>：计算机视觉、异常检测、事件相机、无监督学习、模态融合</p><h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>视频异常检测是计算机视觉领域中的一个核心研究方向，其主要目的是从视频数据中识别出异常事件。随着现代社会对安全与监控需求的日益增加，视频监控系统已被广泛应用于各种场景中，成为确保公共安全和财产安全的重要手段。例如，在城市交通管理中，视频监控不仅用于监控车辆流量，还用于检测交通事故、违法行为等异常事件；在工业制造中，视频监控被用于发现生产过程中的故障和危险操作；在医疗健康领域，视频监控可用于监测患者的行为和状态，及时发现潜在的紧急情况。因此，如何通过视频数据自动检测这些异常行为成为一个至关重要的问题。</p><p>传统的基于帧捕捉的相机虽能完成视频监控的任务，但在处理复杂动态场景时存在一些不足，尤其是在面对快速变化的场景和环境时，常规相机的帧率限制可能导致信息丢失或延迟。随着技术的发展，事件相机作为一种新型视觉传感器，逐渐进入了视频异常检测的研究视野。与传统相机不同，事件相机并不依赖固定的帧率进行图像捕捉，而是仅记录像素亮度的变化事件。这种“事件驱动”的捕捉方式使其具备了高动态范围、低延迟以及低功耗的特点，使得事件相机在处理高速动态变化的场景时表现尤为出色。</p><p>事件相机的这些特点，使其特别适合用于视频异常检测任务。与传统相机相比，事件相机不仅可以更加高效地捕捉快速发生的异常事件，还能够在低光照、强对比等极端环境下保持较高的检测精度。这对于那些需要在实时监控中快速做出反应的应用场景，如安防系统、自动驾驶、工业机器人监控等，具有极大的潜在价值。因此，基于事件相机的视频异常检测技术正在迅速崛起，成为该领域研究的一个重要方向。</p><p>随着事件相机技术的不断发展，它为视频异常检测任务提供了新的思路和解决方案。事件相机不仅克服了传统相机的局限性，还显著提高了检测的效率和准确性。因此，越来越多的研究者开始关注如何将事件相机应用于视频异常检测，并探索其在各种复杂动态场景中的应用潜力。</p><h2 id="事件相机简介"><a href="#事件相机简介" class="headerlink" title="事件相机简介"></a>事件相机简介</h2><p>事件相机与传统相机的主要区别在于其工作原理。传统相机以固定的帧率捕捉整个场景的图像，而事件相机仅记录像素亮度的变化，即“事件”。这种工作方式使得事件相机具有低延迟、高动态范围和低功耗的特点，极大地提高了对动态场景捕捉的效率和准确性。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/事件相机.jpg" alt="使用V2E对ShanghaiTech数据集进行事件化示例"></p><h2 id="常用于视频异常检测的方法"><a href="#常用于视频异常检测的方法" class="headerlink" title="常用于视频异常检测的方法"></a>常用于视频异常检测的方法</h2><p>目前常用于视频异常检测的方法分为监督学习和无监督学习两大类。</p><h3 id="监督学习"><a href="#监督学习" class="headerlink" title="监督学习"></a><strong>监督学习</strong></h3><p>在监督学习方法中，模型的训练依赖于预先标注的数据集，即每个视频片段或帧都被明确地标注为“正常”或“异常”。这种方法的发展基于分类和回归两种主要思路。基于分类的方法通过将所有数据样本映射到不同的类别标签中，即模型根据输入的视频特征对其进行分类，判断是否属于异常行为。基于回归的方法则是将每个数据样本映射到一个连续的异常分数空间，模型会根据输入特征计算出一个异常分数，分数越高意味着该样本越可能是异常。</p><p>尽管监督学习方法在一些特定应用场景中表现出色，但它的一个主要问题在于对大规模、精确标注数据的依赖。由于视频异常检测通常涉及多个领域，异常事件的类型和表现形式差异很大，因此一个能够涵盖所有异常情况的完整数据集难以构建。更为复杂的是，标注数据的成本非常高，特别是在长时间的视频监控场景中，人工标注每个异常事件几乎不可能实现。此外，粗粒度的标签也会影响模型的表现，因为它们无法提供足够细致的异常信息，这可能导致模型误分类或漏检。</p><h3 id="无监督学习"><a href="#无监督学习" class="headerlink" title="无监督学习"></a><strong>无监督学习</strong></h3><p>与监督学习方法不同，无监督学习方法不依赖于预先标注的数据，而是试图通过从未标注数据中自动发现异常模式。在无监督学习中，主要分为基于重建和基于预测两大类方法。</p><p>基于<strong>重建</strong>的方法假设模型在训练过程中通过学习正常行为特征，能够对正常数据进行较为准确的重建，但对于异常数据的重建则会产生较大的误差。因此，模型可以通过在测试阶段计算重建误差来判断是否为异常数据。当重建误差超过一定阈值时，数据被判定为异常。这种方法不依赖标签，且在正常数据较为丰富的情况下效果较好。</p><p>基于<strong>预测</strong>的方法认为，正常视频中的帧与帧之间存在某种规律性的上下文关系，模型可以通过学习这种关系来预测未来帧。当模型无法准确预测未来帧，或者预测误差较大时，意味着该段视频可能包含异常行为。预测模型通常会基于过去的视频帧来推测未来帧的特征，异常帧因为违背了正常帧之间的依赖关系，导致模型难以做出准确预测，从而使得预测误差增大。</p><p>无监督学习方法的一个显著优势是其对数据标签的依赖较小，因此在标签匮乏或无法准确标注的情况下表现良好。然而，这类方法也存在一些缺陷。例如，当模型的泛化能力过强时，可能会过度拟合正常和异常数据，导致异常数据也被准确重建或预测，进而降低检测准确率。此外，某些正常但较为少见的数据样本，可能因其独特性而被模型误判为异常。因此，尽管无监督学习方法在当前视频异常检测研究中占据主流地位，但仍然面临着挑战，需要进一步提升模型的鲁棒性和准确性。</p><h2 id="亟待解决的问题"><a href="#亟待解决的问题" class="headerlink" title="亟待解决的问题"></a>亟待解决的问题</h2><p>尽管已有无数研究者投身于视频异常检测领域的研究，该领域目前仍然面临一系列挑战</p><p><strong>1. 场景性</strong>：视频中的异常事件通常不是孤立的，而是与特定场景、环境或真实情境相关联。这意味着异常的定义和判断需要考虑视频所处的具体场景。例如，在某场景正常的行为，在另一场景就为异常，解决异常的场景依赖问题是提高模型泛化能力的关键。</p><p><strong>2. 模糊性</strong>：异常检测被广泛认为是检测在特定情况下预期不会出现的事件的过程。然而，在现实世界中，正常和异常之间的边界没有明确划分。例如，一些正常样本也会表现出异常事件所具有的奇怪特征，这阻碍了模型的检测精度。</p><p><strong>3. 多样性</strong>：现实世界中的异常行为多种多样，无法完全说明，有时甚至可能尚未发生。因此，在一个数据集中考虑所有可能的异常类型是不切实际的。那么如何让模型在将众多未见的或正常或异常的行为区分开来是视频异常检测领域的一大难题。</p><p><strong>4. 稀疏性</strong>：由于行为的种类繁多，视频异常检测领域的数据集通常有单个行为的数据过少，异常样本明显少于正常样本的不平衡等问题。</p><p><strong>5. 噪声</strong>：在视频异常检测中，监控画面常作为数据集，但这种数据集中的信息密度过低，而且对于长时间视频的标签标注是一个十分耗时的过程。对于一个粗粒度的标签，模型还很有可能错误的学习到了噪声与标签的关系，数据中的高噪声无疑影响着任务的性能。</p><p><strong>6. 隐私性</strong>：监控等视频数据具有隐私性，这限制了视频异常检测在某些领域的数据获取。</p><h2 id="最新解决方法"><a href="#最新解决方法" class="headerlink" title="最新解决方法"></a>最新解决方法</h2><p>解决场景相关问题：</p><p><strong>Sun, S.</strong><a href="#1">[1]</a>等人要解决的关键问题是视频异常检测中的场景性问题即<strong>如何提高场景感知能力，以便检测出场景相关的异常事件</strong>。并应对正常的多样性，让模型学习到不同的正常现象。</p><p>解决思想是利用预训练的视频解析网络提取前景对象和背景场景的高层语义特征，然后利用场景感知的自编码器和层次语义对比学习来学习其中的特征和之间的关系。在测试时，根据输入视频的语义类别，检索高相关的正常特征进行重建，重建误差较高的片段被检测为异常。</p><p>实现方法：</p><ul><li>视频解析：利用预训练的视频解析模型，将前景对象和背景场景的特征分为不同的语义类别。具体使用YOLOv3和FairMOT检测和跟踪对象，使用ViT和PoseConv3D提取对象的外观和动作特征，使用DeepLabV3+生成背景场景的分割图，并使用DBSCAN进行场景聚类。</li><li>层次语义对比：将每个对象的外观或运动特征与对应的场景特征结合起来，形成场景—外观或场景-运动特征，然后用自编码器进行编码和重建。在编码过程中，引入层次语义对比学习，使得编码后的潜在特征在同一语义类别内紧凑，在不同语义类别间分离。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20241010153155488.png" alt="外貌和运动特征在场景中进行对比学习"></li><li>语义特征重建：在测试时，从外部记忆库中检索和加权正常特征来重建测试视频中的对象特征，并根据重建误差来判断异常。重建误差越大，异常得分越高。</li><li><p>动作增强：为了处理稀有但正常的活动，设计了一个基于骨架的增强方法，通过空间变换和时间裁剪来生成更多的正常和异常样本，进一步训练一个二分类器来提升性能。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20241010153613993.png" alt="基于骨骼的运动增强，包括空间变换和时间切割"></p></li></ul><p>研究贡献是提出了一种层次语义对比方法，有效地利用了预训练网络提供的高层语义信息，让计算机更好地理解和表示正常视频中出现的场景和对象，以及它们之间的关系，提高了正常模式的表示和判别能力。设计了一种场景感知的自编码器结构，结合了背景场景和前景对象的信息，同时减少了重建过程中的背景噪声。和设计了一种基于骨架的动作增强方法，增加了稀有活动的样本数量，帮助处理正常模式的不平衡问题。不足之处在于模型中的某些模块也可以用其他更先进的模块替换，例如使用另一种更好的背景解析模型来代替简单的分割图以区分背景的方法。</p><p><strong>Cao, C.</strong><a href="#2">[2]</a>等人想要解决的关键问题是<strong>半监督视频异常检测和异常预测，特别是场景依赖的异常</strong>，给出了另一种解决场景性的方案。</p><p>解决思想是利用前向和后向帧预测模型来估计当前和未来帧的异常分数，同时利用场景条件的变分自编码器来处理场景依赖的异常。</p><p>实现方法是设计了一个前向-后向场景条件自编码器（FBSCAE），包括一个前向网络和一个后向网络，分别用于前向和后向帧预测。每个网络都是一个三层U-Net，包含了条件变分自编码器（CVAE），将场景图像作为输入条件，引导输入帧的特征与场景相关（处理与场景有关的异常）。在训练阶段，使用均方误差损失、L1损失和KL散度损失来优化模型。在推理阶段，使用前向预测误差作为视频异常检测（VAD）的分数，使用前向-后向预测误差的最大值作为视频异常预测（VAA）的分数。</p><p>研究贡献是提出了一个新的大规模数据集NWPU Campus，它是目前最大的半监督视频异常检测基准，也是唯一考虑场景依赖异常和视频异常预测的数据集。提出了一个新的视频异常预测任务，旨在提前预测异常事件的发生，这对于异常事件的预警具有重要意义。不足之处在于对于长期的异常预测还有待改进，以及对于低分辨率的数据集表现不佳。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20241010161712530.png" alt="NWPU Campus dataset"></p><p>生成虚拟异常增强数据集：</p><p><strong>Liu, Z.</strong><a href="#3">[3]</a>等人想要解决的关键问题是<strong>提高模型对于异常判断的泛化能力</strong>。由于异常事件的稀疏性和多样性，目前的模型往往难以泛化到未见过的异常类型。</p><p>解决思想是提出一种基于提示的特征映射框架（PFMF），通过在特征层面进行正常特征到异常特征的映射，来生成数据集中未见过的异常类型。同时，引入了一个异常提示来指导映射的方向，使得生成的异常具有无界的多样性。此外，还设计了一个映射适应分支，通过异常分类器和域分类器来缩小场景差距，使得生成的异常具有场景特异性和一致性。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20241010162250537.png" alt="PFMF框架"></p><p>实现方法是PFMF包含三个部分，即特征提取器、基于提示的特征映射网络和映射适应分支。特征提取器用于将输入的视频实例转换为对应的特征，映射网络用于在同一域内将正常特征映射到异常特征空间，异常提示用于从一个变分自编码器（VAE）中采样，作为映射网络的额外输入，映射适应分支用于对生成的异常特征进行场景适应，包括一个异常分类器和两个域分类器。PFMF的训练过程是统一的，包括特征映射损失、异常分类损失、域分类损失和VAE重构损失。PFMF的推理过程是完全监督的，给定一个未见过的视频实例，通过特征提取器和异常分类器得到实例级别的异常分数，然后通过最大值得到帧级别的异常分数。</p><p>PFMF的优点是能够利用虚拟数据集来生成无界的异常类型，提高了VAD的泛化能力；同时能够通过映射适应分支来缩小虚拟和真实场景之间的差距，提高了VAD的鲁棒性。PFMF的缺点是需要依赖于YOLOv3检测器来提取人物的边界框，这可能会引入一些误差。</p><p>生成伪标签增强数据集：</p><p><strong>Zhang, C.</strong><a href="#4">[4]</a>等人想要解决的关键问题是在弱监督视频异常检测中，<strong>如果数据只有视频级别的标签，如何预测出准确的帧级别的标签。并使用生成出的帧级别的伪标签用于自训练，增强训练效果。</strong></p><p>解决思想是利用完整性和不确定性两个属性来提高伪标签的质量，从而提高异常检测的性能。</p><p>实现方法：</p><ul><li>完整性：设计一个伪标签生成器包含多头分类器，并引入多样性损失， 这样每个头往往会发现不同的异常事件，从而使伪标签生成器覆盖尽可能多的异常事件。</li><li>不确定性：设计一个迭代的不确定性感知的伪标签精炼策略：利用MC Dropout来估计伪标签的不确定性，并根据不确定性选择可靠的样本来训练最终的分类器。</li><li>迭代：使用可靠的样本和可靠的伪标签来训练一个新的帧分类器。用新的帧分类器更新伪标签，并重复上述步骤，直到模型收敛。</li></ul><p>模型优点是能够有效地利用视频中的完整性和不确定性信息，生成高质量的伪标签，从而提高异常检测的准确性和鲁棒性。缺点是需要进行多次迭代和不确定性估计，计算开销较大。</p><p><strong>Mastan, I. D.</strong><a href="#5">[5]</a>等人想要解决的关键问题是<strong>如何在不使用训练数据的情况下，实现图像恢复和图像重定向的任务</strong>，即从单张图像中学习有效的图像特征，并生成不同尺寸或比例的目标图像。解决这个问题可以提高基于重建的视频异常检测任务的性能。</p><p>解决思想是利用深度网络结构作为隐含的图像先验，结合内部学习和上下文特征学习的方法，构建一个通用的框架，通过最小化源图像和目标图像在不同特征表示下的差异，来实现图像恢复和图像重定向。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20241010164645927.png" alt="DCIL框架"></p><p>实现方法是提出了深度上下文内部学习（DCIL）框架，包括以下几个模块：</p><ul><li>网络构建模块：使用一个编码器-解码器结构的生成器网络和一个多尺度补丁判别器网络，根据不同的任务设置网络层、跳跃连接、级联输入和残差块等组件。</li><li>损失函数模块：使用三种损失函数来优化生成器网络，分别是上下文损失（LCL）、对抗损失（LGAN）和重建损失（LR）。上下文损失用于增强生成图像的上下文特征，对抗损失用于匹配源图像和生成图像的补丁分布，重建损失用于保留源图像的全局特征。</li><li>应用模块：根据不同的应用场景，如去噪超分辨率（DSR）、超分辨率（SR）和图像重定向（IR），设置不同的缩放因子、网络参数和损失函数权重，来实现从单张图像中生成不同尺寸或比例的目标图像。</li></ul><p>DCIL框架的优点是不需要任何训练数据，只利用单张图像中的内部信息来学习图像特征，避免了训练数据集的限制和偏差。并且DCIL框架可以适应多种图像恢复和图像重定向的任务，只需要调整网络结构和损失函数的设置，就可以实现不同的目标。论文还将网络结构和损失函数进行了模块化设计，便于调整。DCIL框架的缺点是依赖于单张图像中的自相似性作为先验信息，当源图像中存在高度噪声或低相关性时，可能无法学习到有效的图像特征，并导致生成质量下降。</p><p><strong>Lv, H.</strong><a href="#6">[6]</a>等人想要解决的关键问题是<strong>弱监督视频异常检测</strong>（WSVAD），即利用只有视频级别的二元异常标签（正常或异常）来训练一个片段级别的异常检测器。<strong>需要克服多实例学习（MIL）中的假警报和上下文偏差问题（视频级分类正确但帧级分类错误）</strong>。</p><p>解决思想是提出一个无偏差的多实例学习（UMIL）框架，通过寻求不同上下文偏差的片段之间的不变性来学习无偏差的异常特征。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20241010163506908.png" alt="UMIL框架"></p><p>实现方法：</p><ul><li>划分片段：根据当前的异常检测器f，将视频片段分为两个集合：可信片段集C和模糊片段集A。C中的片段是明显正常或异常的，A中的片段是不确定的。</li><li>聚类模糊片段：利用一个无监督的聚类头g，将A中的片段特征分为两个簇，以区分正常和异常片段。g通过最小化预测相似度的二元交叉熵损失来训练。（利用dot-product进行无监督的二元分类，分类后的信息会作为第三步的监督）</li><li>训练异常检测器：在C和A上同时训练异常检测器f，使其能够预测C中的二元标签，以及分离A中的两个簇（由第二步生成）。这样可以消除C中的上下文偏置，并学习无偏置的异常特征。</li></ul><p>UMIL的优点是能够利用模糊片段来消除可信片段中的上下文偏差，提高WSVAD的性能和鲁棒性。能够将特征表示微调和异常检测器学习整合到一个端到端的训练方式中，得到一个更适合VAD的特征表示。并且采用了一种细粒度的视频划分策略（将每个视频划分为长度为一秒的片段，而不是使用每个粗粒度片段的平均特征作为分类器的输入），保留了视频片段中微妙的异常信息。UMIL的缺点是依赖于无监督聚类来区分模糊片段，其性能受到聚类算法和参数选择的影响。需要预先训练一个MIL模型来初始化异常检测器，并且需要追踪每个片段的预测历史来划分可信和模糊片段，增加了计算开销。</p><p><strong>Yang, Z.</strong><a href="#7">[7]</a>等人想要解决的关键问题是<strong>如何将视频中更高层次的视觉特征和综合的时空关系用于视频异常检测任务中</strong>。</p><p>解决思想是提出一种全新（不同于帧重建、帧预测等）的视频异常检测方法：基于关键帧恢复视频事件的方法。该方法鼓励DNN根据包含隐含的外观和运动关系的视频关键帧来推断缺失的多帧，从而恢复视频事件，这可以更有效地激励DNN挖掘和学习视频中潜在的高层次视觉特征和综合时空变化关系。</p><p>实现方法是提出了一种新颖的U形Swin Transformer网络（USTN-DSC），其中引入了一个交叉注意力和一个时域上采样残差跳跃连接来进一步辅助恢复视频中复杂的静态和动态运动对象特征。此外，还提出了一种简单有效的相邻帧差分损失来约束视频序列的运动一致性。</p><p>模型优点是能够更好地捕捉视频中长距离的时空依赖关系，提高对异常事件的敏感性和区分度。不足之处是需要更多的计算资源和训练时间，以及对不同场景和运动模式的泛化能力还有待提高。</p><p>Event数据中的特征提取工具：</p><p><strong>Peng, Y.</strong><a href="#8">[8]</a>等人想要解决的关键问题是如何<strong>利用Transformer网络来提取事件相机数据中的空间、时间和极性信息</strong>，从而提高事件视觉任务的性能。</p><p>解决思想是提出一种新的事件表示方法，称为Group Token，将异步事件根据时间戳和极性进行分组，并设计一个新的Transformer网络，称为Group Event Transformer (GET)，在Group Token上进行有效的特征提取和整合。</p><p>实现方法：GET包括三个主要模块：Group Token Embedding (GTE)、Event Dual Self-Attention (EDSA) block和Group Token Aggregation (GTA) module。GTE将事件流转换为Group Token，EDSA block在空间和时间—极性维度上进行局部自注意力操作，并建立双重残差连接，提取事件相机数据的空间和时间—极性特征，GTA module利用重叠分组卷积来实现两个维度的信息整合和解耦。</p><p>GET的优点是能够充分利用事件数据的特性，提高事件视觉任务的性能，同时具有较低的计算成本和模型大小。GET的缺点是需要根据不同的数据集和任务来调整Group Token的生成参数，以达到最佳效果。</p><h2 id="事件相机对视频异常检测的帮助"><a href="#事件相机对视频异常检测的帮助" class="headerlink" title="事件相机对视频异常检测的帮助"></a>事件相机对视频异常检测的帮助</h2><ol><li>利用事件相机的低数据率、低能耗的优点降低异常检测的模型训练成本和应用的成本。</li><li>事件相机能大大减少视频中的隐私信息，解决异常检测的数据和应用中存在的隐私方面的问题。</li><li>利用事件相机的高动态范围的优点可以提高模型在低/高亮度等画面下的健壮性。</li><li>由于事件相机对变化的敏感性，可以借助对变化敏感事件相机数据对异常检测进行辅助。<strong>Liu, Z.</strong><a href="#9">[9]</a>等人就利用了事件相机来辅助交通领域的物体检测，并且发现事件相机在动态物体检测贡献较大。</li></ol><h2 id="事件相机与RGB相机的模态融合方法"><a href="#事件相机与RGB相机的模态融合方法" class="headerlink" title="事件相机与RGB相机的模态融合方法"></a>事件相机与RGB相机的模态融合方法</h2><p><strong>Yang, Y.</strong><a href="#10">[10]</a>等人提出了一个用于事件引导HDR视频重建的多模态学习框架。为了更好地利用两种视觉信号模态对同一场景的了解，该文提出一种学习共享潜在空间的多模态表示对齐策略，以及针对不同区域不同动态范围对两类信号进行互补的融合模块。并且利用时间相关性来抑制重建的HDR视频中的闪烁效果。</p><p>实现方法是将事件相机与RGB相机的数据投影到共享表示空间上，使得两种模态的数据对齐。再使用置信度引导的多模态融合模块，分步执行模态间重建和模态内重建，最后将两个编码器和 HDR 解码器联合训练。</p><p><strong>Zhu, Z.</strong><a href="#11">[11]</a>等人基于预训练的ViT框架，鼓励ViT弥合两种模式之间的巨大分布差距，实现全面的跨模式信息交互，从而增强其能力。</p><p>实现方法是提出一种掩码建模策略，该策略随机屏蔽某些token的特定模态，以强制来自不同模态的token主动交互。还提出了一个正交高秩损失来正则化注意力矩阵，用于抑制跨模态掩码引起的图像闪烁，同时放大其积极作用。</p><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>在完成了对视频异常检测领域的深入探讨与研究后，我们可以看到，尽管在这一领域已经取得了显著进展，但仍有许多挑战和未解决的问题等待着科研工作者们的攻克。我们认为在视频异常检测任务中，加入事件相机是一个新颖的创新点。由于其独特的特点，在提高视频异常检测准确性和效率上都将发挥很大的作用。</p><p>通过广泛阅读论文，明确了视频异常检测领域的常用方法和亟待解决的问题后，我们开始思考如何在现有的研究上加入事件相机。为此我们进一步的翻阅了相关论文，总结目前的模态融合方法，寻找适合两种模态的编解码器，着手于模型搭建和进一步的实验设计于验证。</p><p>在未来的工作中，我们希望能够找到更为有效的解决方案，以应对视频异常检测领域所面临的挑战。最终，我们相信，通过不断的努力和探索，视频异常检测的研究将为我们带来更安全、更智能的生活环境。</p><h2 id="参考文献"><a href="#参考文献" class="headerlink" title="参考文献"></a><strong>参考文献</strong></h2><p><code><a id="1"></code>[1]<code></a></code> Sun, S., & Gong, X. (2023). Hierarchical Semantic Contrast for Scene-Aware Video Anomaly Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22846–22856.</p><p><code><a id="2"></code>[2]<code></a></code> Cao, C., Lu, Y., Wang, P., & Zhang, Y. (2023). A New Comprehensive Benchmark for Semi-Supervised Video Anomaly Detection and Anticipation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20392–20401.</p><p><code><a id="3"></code>[3]<code></a></code>Liu, Z., Wu, X., Zheng, D., Lin, K., & Zheng, W. (2023). Generating Anomalies for Video Anomaly Detection With Prompt-Based Feature Mapping. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24500–24510.</p><p><code><a id="4"></code>[4]<code></a></code> Zhang, C., Li, G., Qi, Y., et al. (2023). Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16271–16280.</p><p><code><a id="5"></code>[5]<code></a></code> Mastan, I. D., & Raman, S. (2020). DCIL: Deep Contextual Internal Learning for Image Restoration and Image Retargeting. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2366–2375.</p><p><code><a id="6"></code>[6]<code></a></code> Lv, H., Yue, Z., Sun, Q., et al. (2023). Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8022–8031.</p><p><code><a id="7"></code>[7]<code></a></code> Yang, Z., Liu, J., Wu, Z., et al. (2023). Video Event Restoration Based on Keyframes for Video Anomaly Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14592–14601.</p><p><code><a id="8"></code>[8]<code></a></code> Peng, Y., Zhang, Y., Xiong, Z., et al. (2023). GET: Group Event Transformer for Event-Based Vision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6038–6048.</p><p><code><a id="9"></code>[9]<code></a></code> Liu, Z., Yang, N., Wang, Y., Li, Y., Zhao, X., & Wang, F. (2023). Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion. ArXiv, abs/2311.00436.</p><p><code><a id="10"></code>[10]<code></a></code> Yang, Y., Han, J., Liang, J., et al. (2023). Learning Event Guided High Dynamic Range Video Reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13924–13934.</p><p><code><a id="11"></code>[11]<code></a></code> Zhu, Z., Hou, J., & Wu, D. O. (2023). Cross-Modal Orthogonal High-Rank Augmentation for RGB-Event Transformer-Trackers. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22045–22055</p>]]></content>
<categories>
<category> 科研日记 </category>
</categories>
<tags>
<tag> 人工智能 </tag>
<tag> 综述 </tag>
<tag> 计算机视觉 </tag>
<tag> 视频异常检测 </tag>
</tags>
</entry>
<entry>
<title>如何快速让个人博客被搜索引擎收录</title>
<link href="/2024/03/13/%E5%A6%82%E4%BD%95%E5%BF%AB%E9%80%9F%E8%AE%A9%E4%B8%AA%E4%BA%BA%E5%8D%9A%E5%AE%A2%E8%A2%AB%E6%90%9C%E7%B4%A2%E5%BC%95%E6%93%8E%E6%94%B6%E5%BD%95/"/>
<url>/2024/03/13/%E5%A6%82%E4%BD%95%E5%BF%AB%E9%80%9F%E8%AE%A9%E4%B8%AA%E4%BA%BA%E5%8D%9A%E5%AE%A2%E8%A2%AB%E6%90%9C%E7%B4%A2%E5%BC%95%E6%93%8E%E6%94%B6%E5%BD%95/</url>
<content type="html"><![CDATA[<h2 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h2><p>当我们好不容易搭建好自己的个人博客时，在搜索引擎搜索时才发现，无论是搜索网站的标题还是网址都无法搜索到，搜索不到就代表着没有流量、没人看你写的文章，这可咋办。难道辛辛苦苦搭建的网站只有知道网址的人才能访问到吗，如何打破这信息孤岛呢？下面交给你解决办法，把你的网站主动推荐给各大搜索引擎让搜索引擎收录。</p><p>由于各大搜索引擎的本质就是一个爬虫在不断爬取互联网上的内容，也许你的网站在不经意间已经被收录了。</p><p>你可以通过在搜索引擎中输入</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">site:<域名> （e.g. site:xiaobaozi.cn）</span><br></pre></td></tr></table></figure><p>若可以搜索出你的网站，则说明已经被收录。</p><p>不过你仍可以进行接下来的步骤，在各大搜索引擎的后台添加你的网站并绑定你的账号，之后你就可以在后台看到相关的流量统计和相关增加曝光度的操作。</p><h2 id="百度收录"><a href="#百度收录" class="headerlink" title="百度收录"></a>百度收录</h2><p>若未被收录，可以点击<a href="https://ziyuan.baidu.com/linksubmit/url?sitename">提交网址</a></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/百度未收录.png" alt=""></p><p>进行连接提交（这是搜索引擎用户对搜索不到的网址进行反馈提交，无法保证收录）</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20240314001014339.png" alt=""></p><p>若要进一步提交网站可以选择</p><h3 id="添加站点到百度"><a href="#添加站点到百度" class="headerlink" title="添加站点到百度"></a>添加站点到百度</h3><p>我们需要登录<a href="https://ziyuan.baidu.com/">百度搜索资源平台</a>，登录成功之后在上方用户中心→<a href="https://ziyuan.baidu.com/site">站点管理</a>中点击添加网站，输入域名，按照步骤走。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20240314001927371.png" alt=""></p><p>进入第三步验证您对网站的所有权，可以选择文件验证或HTML标签认证。</p><h4 id="文件验证"><a href="#文件验证" class="headerlink" title="文件验证"></a>文件验证</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/文件验证.png" alt=""></p><ol><li>下载验证文件，放到指定目录下。以hexo搭建的博客为例，放到 <code>博客目录/public</code>目录下。</li><li>执行 <code>hexo cl && hexo g && hexo d</code>命令即可。</li><li>等待网站更新，在你的网站域名后添加验证文件的文件名（e.g. <code>https://www.xiaobaozi.cn/baidu_verify_codeva-<xxxxx>.html</code>)，若能访问到一串字符串说明成功。（仅出现一串字符串在左上角，其余为空白，无其他内容）</li><li>成功后点击 <code>完成验证</code>即可</li></ol><h3 id="向百度推送网站的资源"><a href="#向百度推送网站的资源" class="headerlink" title="向百度推送网站的资源"></a>向百度推送网站的资源</h3><p>经过上面的步骤，百度已经知道有我们网站的存在了，但是百度还不知道我们的网站上有什么内容，所以要向百度推送我们的内容。还是在百度资源搜索平台上方点击搜索服务→资源提交→<a href="https://ziyuan.baidu.com/linksubmit/index">普通收录</a></p><p>资源提交的方式有多种</p><p><a href="https://ziyuan.baidu.com/college/courseinfo?id=267&page=3#h2_article_title8">如何选择普通收录方式</a></p><ul><li><strong>API推送：</strong>最为快速的提交方式，建议您将站点当天新产出链接立即通过此方式推送给百度，以保证新链接可以及时被百度收录。</li><li><strong>sitemap：</strong>您可以定期将网站链接放到Sitemap中，然后将Sitemap提交给百度。百度会周期性的抓取检查您提交的Sitemap，对其中的链接进行处理，但收录速度慢于API推送。</li><li><strong>手动提交：</strong>如果您不想通过程序提交，那么可以采用此种方式，手动将链接提交给百度。</li></ul><h4 id="API提交"><a href="#API提交" class="headerlink" title="API提交"></a>API提交</h4><p>API提交首先需要安装插件</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">npm install hexo-baidu-url-submit --save</span><br></pre></td></tr></table></figure><p>然后在 hexo 根目录配置文件 <code>_config.yml</code> 中，添加：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 主动推送百度，被百度收录</span></span><br><span class="line"><span class="attr">baidu_url_submit:</span></span><br><span class="line"> <span class="attr">count:</span> <span class="number">10</span> <span class="comment"># 提交最新的10个链接</span></span><br><span class="line"> <span class="attr">host:</span> <span class="comment"># 百度站长平台中注册的域名</span></span><br><span class="line"> <span class="attr">token:</span> <span class="comment"># 密钥，百度站长平台 > 普通收录 > 推送接口 > 接口调用地址中token字段</span></span><br><span class="line"> <span class="attr">path:</span> <span class="string">baidu_urls.txt</span> <span class="comment"># 文本文档的地址， 新链接会保存在此文本文档里，不用改</span></span><br></pre></td></tr></table></figure><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/百度.png" alt="百度"></p><p>其次，记得查看 hexo 根目录中 <code>_config.yml</code> 文件中 <code>url</code> 的值， 必须包含是百度站长平台注册的域名。</p><p>最后，在 <code>_config.yml</code> 文件中的 <code>deploy</code> 加入新的 <code>type</code>:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">deploy:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">type:</span> <span class="string">git</span></span><br><span class="line"> <span class="attr">repository:</span> <span class="string">git@github.com:Warma10032/Warma10032.github.io.git</span></span><br><span class="line"> <span class="attr">branch:</span> <span class="string">main</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">type:</span> <span class="string">baidu_url_submitter</span></span><br></pre></td></tr></table></figure><div class="note warning modern"><p>这里是新建一个 type，一定要注意这段代码里面各行的缩进值</p></div><p>执行 <code>hexo cl && hexo g && hexo d</code>命令即可。</p><p>后续慢慢等收录吧，百度收录比较慢。</p><h4 id="Sitemap"><a href="#Sitemap" class="headerlink" title="Sitemap"></a>Sitemap</h4><div class="note danger modern"><p>目前百度正在清理陈旧sitemap文件，并且关闭了个人小站的添加sitemap的入口。对于有域名有备案的网站可以登录百度查看是否可以提交sitemap文件。</p></div><div class="note info modern"><p>Sitemap（站点地图）是一种文件的统称，通常Sitemap（站点地图）可以是txt或者XML格式。通过Sitemap（站点地图）你可以告诉搜索引擎关于你的站点中的网页、视频或者其他文件的相关信息，帮助搜索引擎更好的认识和理解你的站点。格式正确的Sitemap（站点地图）文件会帮助搜索引擎更高效地抓取你的网站。</p></div><p>通过sitemap方式推送我们首先需要生成一个站点地图。</p><p>安装百度和 Google 的站点地图生成插件：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">npm install hexo-generator-sitemap --save </span><br><span class="line">npm install hexo-generator-baidu-sitemap --save</span><br></pre></td></tr></table></figure><p>然后来到 hexo 根目录配置文件 <code>_config.yml</code>，在下面添加：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 站点地图</span></span><br><span class="line"><span class="attr">sitemap:</span></span><br><span class="line"> <span class="attr">path:</span> <span class="string">sitemap.xml</span></span><br><span class="line"><span class="attr">baidusitemap:</span></span><br><span class="line"> <span class="attr">path:</span> <span class="string">baidusitemap.xml</span></span><br></pre></td></tr></table></figure><p>安装完成后我们执行 <code>hexo cl && hexo g</code>命令后我们会发现在 <code>public</code>目录下面多了 <code>baidusitemap.xml</code>和 <code>sitemap.xml</code>文件。 我们打开文件可以看到生成的其实就是我们每篇文章的url。</p><p>然后重新推送到服务器，访问如下 URL:</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">https://你的域名/sitemap.xml</span><br><span class="line">https://你的域名/baidusitemap.xml</span><br></pre></td></tr></table></figure><p>看看网页中有没有内容，有的话就成功。</p><p>最后将 <code>https://你的域名/baidusitemap.xml</code>填入并提交即可。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20240328125839744.png" alt=""></p><div class="note primary modern"><p>Google Sitemap提交 <code>https://你的域名/sitemap.xml</code>，方法类似，不再赘述。</p></div><h4 id="手动提交"><a href="#手动提交" class="headerlink" title="手动提交"></a>手动提交</h4><p>即手动将你的连接输入并提交。</p><h2 id="谷歌收录"><a href="#谷歌收录" class="headerlink" title="谷歌收录"></a>谷歌收录</h2><p>提交谷歌搜索引擎比较简单，在提交之前，我们依然可以使用 <code>site:域名</code> 查看网站是否被收录。进入 <a href="https://developers.google.com/search#?modal_active=none">Google 搜索中心</a>，登录你的谷歌账号。然后找到<a href="https://search.google.com/search-console/welcome">注册 Search Console</a>(在 “使用入门–>SEO 新手指南” 中可以找到入口)，就直接输入你要收录的网站域名就行。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20240328131702702.png" alt=""></p><h3 id="网域"><a href="#网域" class="headerlink" title="网域"></a>网域</h3><p>输入你的域名后进行DNS验证。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/20250125135930720.png" alt=""></p><p>以腾讯云为例。</p><p>进入域名解析添加解析记录即可。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20240328184907173.png" alt=""></p><h3 id="网址前缀"><a href="#网址前缀" class="headerlink" title="网址前缀"></a>网址前缀</h3><p>按示例格式输入你的网址前缀后，Google会收录所有以该前缀开头的网址。</p><p>需要进行网站所有权验证，选HTML文件验证，与百度收录类似。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/image-20240328191231313.png" alt=""></p><ol><li>下载验证文件，放到指定目录下。以hexo搭建的博客为例，放到 <code>博客目录/public</code>目录下。</li><li>执行 <code>hexo cl && hexo g && hexo d</code>命令即可。</li><li>等待网站更新，在你的网站域名后添加验证文件的文件名（e.g. <code>https://www.xiaobaozi.cn/google<xxxxx>.html</code>)，若能访问到一串字符串说明成功。（仅出现一串字符串在左上角，其余为空白，无其他内容）</li><li>成功后点击 <code>验证</code>即可</li></ol><h3 id="站点地图"><a href="#站点地图" class="headerlink" title="站点地图"></a>站点地图</h3><p>Google收录速度很快，对站点地图也不是刚需。不过也可以在对应页面上传你的站点地图文件，便于Google爬取。</p><h2 id="必应收录"><a href="#必应收录" class="headerlink" title="必应收录"></a>必应收录</h2><p>必应收录也是很简单，点击<a href="https://www.bing.com/webmasters/about">必应站长</a>。先注册登录，必应收录有两种方式，一种使用刚刚谷歌导入过去，第二种是就是自己添加 URL，方法与Google收录类似，不再赘述。</p>]]></content>
<categories>
<category> 经验之谈 </category>
</categories>
<tags>
<tag> 博客 </tag>
<tag> hexo </tag>
<tag> SEO </tag>
<tag> 搜索引擎 </tag>
</tags>
</entry>
<entry>
<title>虚拟定位完成校园跑</title>
<link href="/2024/03/07/%E8%99%9A%E6%8B%9F%E5%AE%9A%E4%BD%8D%E5%AE%8C%E6%88%90%E6%A0%A1%E5%9B%AD%E8%B7%91/"/>
<url>/2024/03/07/%E8%99%9A%E6%8B%9F%E5%AE%9A%E4%BD%8D%E5%AE%8C%E6%88%90%E6%A0%A1%E5%9B%AD%E8%B7%91/</url>
<content type="html"><![CDATA[<h2 id="原理"><a href="#原理" class="headerlink" title="原理"></a>原理</h2><p>对于利用定位检测的校园跑软件，使用fake location这个软件实现虚拟定位，进而达到足不出户刷校园跑。</p><h2 id="前期准备"><a href="#前期准备" class="headerlink" title="前期准备"></a>前期准备</h2><p>fake location软件：<a href="https://github.com/Lerist/FakeLocation/releases/">下载地址</a></p><p>有定位功能的root环境（root可提高成功几率）：可以是本机root/面具、带有root的模拟器、带有root的虚拟机。</p><h2 id="演示"><a href="#演示" class="headerlink" title="演示"></a>演示</h2><p>以安卓光速虚拟机为例（光速虚拟机提供免费root功能）</p><ol><li>下载光速虚拟机：<a href="https://gsxnj.cn/index/">下载地址</a></li><li><p>创建安卓7虚拟环境（初次使用按软件要求进行配置，包括进入开发者模式、无线调试等可自行百度）</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/创建安卓7虚拟环境.jpg" style="zoom:50%;" /></p></li><li><p>导入fake location和校园跑软件</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/导入.jpg" style="zoom: 50%;" /></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/选择导入.jpg" style="zoom:50%;" /></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/导入软件.jpg" alt="导入软件" style="zoom: 50%;" /></p></li><li><p>选择位置进行位置模拟（以root模式）</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/添加定位.jpg" alt="添加定位" style="zoom:50%;" /></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/选择定位.jpg" alt="选择位置" style="zoom:50%;" /></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/开始模拟.jpg" alt="开始模拟" style="zoom:50%;" /></p></li><li>移动摇杆进行移动（非VIP限速1m/s）</li></ol><h2 id="本校解决方案（东南大学）"><a href="#本校解决方案（东南大学）" class="headerlink" title="本校解决方案（东南大学）"></a>本校解决方案（<del>东南大学</del>）</h2><p>由于本校采用的是间隔时间进行定位采样，可以在目标线路（操场）上定位多个点，再在历史定位中间依次更换虚拟定位让系统记录。</p><h2 id="通用解决方案（提升体验）"><a href="#通用解决方案（提升体验）" class="headerlink" title="通用解决方案（提升体验）"></a>通用解决方案（提升体验）</h2><p>开VIP🤯</p><p>可获得快速摇杆，路线模拟（自动运行）等。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/专业版1.jpg" style="zoom: 50%;" /><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/专业版2.jpg" style="zoom: 50%;" /></p><h2 id="常见失败原因"><a href="#常见失败原因" class="headerlink" title="常见失败原因"></a>常见失败原因</h2><ol><li>未以root模式启动</li><li>需要启动基站模拟等增强功能</li><li>未给fake location足够权限</li><li>未给光速虚拟机足够权限</li></ol>]]></content>
<categories>
<category> 生活趣闻 </category>
</categories>
<tags>
<tag> 校园跑 </tag>
</tags>
</entry>
<entry>
<title>Transformer经典论文阅读后综述</title>
<link href="/2024/02/24/Transformer%E7%BB%8F%E5%85%B8%E8%AE%BA%E6%96%87%E9%98%85%E8%AF%BB%E5%90%8E%E7%BB%BC%E8%BF%B0/"/>
<url>/2024/02/24/Transformer%E7%BB%8F%E5%85%B8%E8%AE%BA%E6%96%87%E9%98%85%E8%AF%BB%E5%90%8E%E7%BB%BC%E8%BF%B0/</url>
<content type="html"><![CDATA[<h1 id="Transformer-in-Deep-Learning"><a href="#Transformer-in-Deep-Learning" class="headerlink" title="Transformer in Deep Learning"></a>Transformer in Deep Learning</h1><h2 id="摘要"><a href="#摘要" class="headerlink" title="摘要"></a>摘要</h2><p>Transformer模型是一种革命性的神经网络架构，它在自然语言处理（NLP）和计算机视觉（CV）等多个领域都取得了卓越的成就。本论文旨在介绍Transformer模型的结构、以及在NLP领域和CV领域中的应用。深入探讨Transformer模型在不同领域之间的共同特点和差异。我们首先详细介绍了Transformer模型的核心结构，然后探讨了在NLP任务中的Transformer变种（如BERT和GPT），最后研究了Transformer在计算机视觉中的新兴应用（如Vision Transformer）。通过本文，读者将能够深入了解Transformer模型的工作原理以及它在不同领域中应用的优势和改进。</p><h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>自然语言处理（NLP）和计算机视觉（CV）一直是人工智能领域的两大关键领域。这些领域的发展一直受制于模型的性能和能力，而Transformer模型的出现冲击了RNN和CNN的统治地位。在过去的几年里，Transformer模型已经成为NLP和CV任务的首选模型之一，且具有统一NLP和CV研究方法的势头。</p><p>首先，我们将详细介绍Transformer模型的基本结构，包括自注意力机制（self-attention）。这种机制使其能够同时处理输入序列中的各个元素，无论是单词、像素还是其他形式的数据。这个思想的强大之处在于它的通用性，它使Transformer能够在各种领域中取得成功。在NLP领域，Transformer模型已在文本分类、命名实体识别和机器翻译等领域取得了巨大成功。而在计算机视觉领域，Transformer也已经在图像分类和目标检测中崭露头角。</p><p>然而，虽然Transformer模型在不同领域之间的应用非常广泛，但它在每个领域中都有自己的独特挑战和特点。例如，在NLP中，BERT模型通过让模型获得双向输入，掌握上下文信息来改善预训练词嵌入，而在CV中，ViT模型通过对图像的分块处理来应对图像数据的巨大复杂性。通过对这些异同之处的深入研究，我们可以更好地理解Transformer模型的多领域适用性，以及如何将其推向新的高度。</p><h2 id="什么是Transformer"><a href="#什么是Transformer" class="headerlink" title="什么是Transformer"></a>什么是Transformer</h2><p>Transformer是一个基于注意力机制的神经网络模型，Transformer模型由编码器和解码器组成，两者都由多个相同的层组成，每一层有两个子层：多头自注意力机制层和全连接前馈神经网络，Transformer模型利用自注意力机制捕捉输入序列中每个元素与其他元素之间的关系，利用多头机制提高模型的性能。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/transformer结构.png" alt="Transformer架构" style="zoom: 67%;" /></p><p>接下来我将自下而上从输入到输出介绍transformer模型。</p><h3 id="Input-Embedding-and-Output-Embedding（输入嵌入和输出嵌入）"><a href="#Input-Embedding-and-Output-Embedding（输入嵌入和输出嵌入）" class="headerlink" title="Input Embedding and Output Embedding（输入嵌入和输出嵌入）"></a>Input Embedding and Output Embedding（输入嵌入和输出嵌入）</h3><p>Word Embedding（词嵌入）是将离散化的符号序列（如单词、字符）转换为连续的向量表示，以便使神经网络可以有效地处理文本数据。它使得模型能够理解文本数据的语义和上下文信息，并为后续的自注意力层和前馈神经网络提供了输入。</p><h3 id="Positional-Encoding（位置编码）"><a href="#Positional-Encoding（位置编码）" class="headerlink" title="Positional Encoding（位置编码）"></a>Positional Encoding（位置编码）</h3><p>由于 Transformer 不包含处理输入序列的循环或卷积操作，它不能自动地理解单词的顺序信息。因此，我们必须注入一些关于相对或绝对位置的信息序列中的标记。为此，我们引入了“位置编码”来将位置信息嵌入到模型中。位置编码是一个与位置、维度相关的矩阵，它会与输入的词嵌入相加。在原论文中，作者采用了正余弦函数来进行位置编码。</p><h3 id="Attention（注意力机制）"><a href="#Attention（注意力机制）" class="headerlink" title="Attention（注意力机制）"></a>Attention（注意力机制）</h3><p>注意力函数可以描述为将查询（query）和一组键值对（key-value pairs）映射到输出，其中查询(Q)、键(K)、值(V)和输出都是向量。 输出被计算为值的加权和，其中分配给每个值的权重是由查询(Q)与相应键(v)的兼容性函数计算的。</p><p><img src="/注意力函数.png" alt="注意力函数"></p><h4 id="Scaled-Dot-Product-Attention（缩放点积注意力）"><a href="#Scaled-Dot-Product-Attention（缩放点积注意力）" class="headerlink" title="Scaled Dot-Product Attention（缩放点积注意力）"></a>Scaled Dot-Product Attention（缩放点积注意力）</h4><p>自注意力机制（Self-attention）是 Transformer 模型的关键组成部分，它允许模型在一个序列中的不同位置之间建立权重连接，从而在一个步骤内同时考虑所有位置的信息。自注意力机制的计算分为以下几步：</p><ul><li>对于一个输入序列，首先通过三个线性变换分别得到查询(Q)、键(K)和值(V)的向量表示。这些向量用于计算注意力分数和生成权重。</li><li>计算注意力分数：为了衡量每个位置与其他位置的关联程度，通过计算查询和键之间的点积，然后进行缩放（通常使用缩放因子，如$\sqrt{d_k}$），得到注意力分数（Attention Scores）。</li><li>注意力分数的归一化（Normalization）：应用 Softmax 函数将注意力分数转化为概率分布，使得每个位置对其他位置的贡献权重为1，形成归一化的注意力权重。</li><li>加权求和：将注意力权重应用于值的向量，然后将所有加权的值进行加和，得到最终的输出。这个输出包含了所有位置的信息，但每个位置的贡献由其与其他位置的关联程度决定。</li></ul><p>自注意力机制允许模型动态地分配不同位置的权重，从而在不同任务中灵活捕获上下文信息。这是 Transformer 模型在各种自然语言处理任务中表现出色的关键之一。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/自注意力机制.png" alt="自注意力机制"></p><h4 id="Multi-Head-Attention（多头注意力机制）"><a href="#Multi-Head-Attention（多头注意力机制）" class="headerlink" title="Multi-Head Attention（多头注意力机制）"></a>Multi-Head Attention（多头注意力机制）</h4><p>Transformer 中的多头注意力机制（Multi-Head Attention）是自注意力机制的扩展，允许模型在不同表示子空间中学习自注意力。这个机制增加了模型的表示能力，使其能够同时关注输入序列中的不同信息，从而更好地捕捉序列中的复杂关系。多头注意力机制通过将多个并行的自注意力机制组合在一起，每个自注意力机制都称为一个“头”。每个头有自己的一组查询、键和值参数，这些参数是通过学习而得的。将所有注意力头的输出连接在一起，并通过一个线性变换来产生最终的多头注意力输出。其优点在于提高了模型的学习能力：不同的头可以学习捕获不同的关系，从而提高了模型的表示能力。例如，一些头可以关注语法关系，而其他头可以关注语义关系。提高了训练效率：多头注意力可以并行计算，因为每个头都是独立的，这提高了模型的训练和推理效率。提高了模型稳定性：多头注意力有助于减轻注意力机制中的一些不稳定性，因为多个头的组合可以平滑化注意力权重的分布。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/多头自注意力机制.png" alt="多头注意力机制"></p><h4 id="Add-amp-Norm（残差连接-amp-层归一化）"><a href="#Add-amp-Norm（残差连接-amp-层归一化）" class="headerlink" title="Add & Norm（残差连接&层归一化）"></a>Add & Norm（残差连接&层归一化）</h4><p>Transformer在每个子层（自注意力层和前馈神经网络层）的输入和输出之间，引入了残差连接和层归一化，残差连接可以通过跳过该层来传递一定量的信息，使梯度直接流过某层；层归一化把数据转化成均值为 0 方差为1的数据，保证数据特征分布的稳定性。这些方法可以帮助缓解训练过程中的梯度消失问题。</p><h4 id="Feed-Forward-Neural-Network（前馈神经网络）"><a href="#Feed-Forward-Neural-Network（前馈神经网络）" class="headerlink" title="Feed-Forward Neural Network（前馈神经网络）"></a>Feed-Forward Neural Network（前馈神经网络）</h4><p>Transformer的每个编码器和解码器层都包括一个前馈神经网络。它将自注意力层的输出映射到一个更高维度的空间，然后再映射回原始维度。这个过程包括两个线性变换和一个非线性激活函数，通常是ReLU。其作用是通过线性变换，先将数据映射到高纬度的空间再映射到低纬度的空间，提取了更深层次的特征。并且加入一定非线性变换，提高模型学习能力。</p><h4 id="Masked-Multi-Head-Attention（带掩码的多头注意力机制）"><a href="#Masked-Multi-Head-Attention（带掩码的多头注意力机制）" class="headerlink" title="Masked Multi-Head Attention（带掩码的多头注意力机制）"></a>Masked Multi-Head Attention（带掩码的多头注意力机制）</h4><p>Masked Multi-Head Attention是一种用于处理可变长度序列的重要技术。它的主要作用是在自注意力计算中，确保模型只关注当前位置之前的信息，而不会泄露未来位置的信息。在处理序列数据时，尤其是在解码器中，我们会用到掩码机制来确保生成的每个位置的信息只取决于当前位置及其之前的信息。这是因为在生成输出序列时，未来的信息是不可见的。具体来说，掩码机制会创建一个掩码矩阵，该矩阵与注意力分数相乘，将未来位置的分数置为负无穷大（或经过 softmax 后为零），从而在注意力计算中消除了未来位置的影响。这使得模型只能关注于当前位置以及之前的信息，而不关注当前位置之后的信息。</p><h3 id="Transformer的优势"><a href="#Transformer的优势" class="headerlink" title="Transformer的优势"></a>Transformer的优势</h3><ul><li>与RNN和CNN相比，Transformer具有更好的并行性：Transformer中的自注意力机制可以并行计算，而RNN和CNN中的循环结构和卷积结构需要依次计算，难以并行化。例如RNN需要从循环结构中获取序列的时间依赖关系，而CNN需要从小区域到大区域依次扩大感受野。</li><li>更好的长距离依赖建模能力：Transformer中的自注意力机制可以捕捉输入序列中任意两个位置之间的关系，而RNN只能捕捉相邻位置之间的关系，CNN只能捕捉局部区域内的关系。</li><li>更好的全局特征获取能力：Transformer中的自注意力机制可以对输入序列中每个位置进行注意力计算，从而获取全局上下文信息，而RNN和CNN只能获取局部上下文信息。鉴于Transformer相较于之前的模型有这么多优点，它很快被用在了NLP，CV等领域上。</li></ul><h2 id="Transformer-in-NLP"><a href="#Transformer-in-NLP" class="headerlink" title="Transformer in NLP"></a>Transformer in NLP</h2><p>Transformer被提出时，是被用在机器翻译上的。它帮助改善了RNN在并行性和长文本处理上的缺陷，使得机器翻译模型的准确率提高了几个百分点。当人们意识到transformer和attention在NLP上具有很多优势，便想把transformer扩展应用文本分类、语义分析、语言生成等领域。他们将transformer 与预训练结合，训练出一个模型，再通过微调去适应不同的任务。这样可以减少模型在特定任务上的训练时间和数据量，并且降低过拟合的发生模型更加准确和可靠。于是在NLP中，预训练transformer开始大行其道。</p><h3 id="Bert-Bidirectional-Encoder-Representation-from-Transformers"><a href="#Bert-Bidirectional-Encoder-Representation-from-Transformers" class="headerlink" title="Bert: :Bidirectional Encoder Representation from Transformers"></a>Bert: :Bidirectional Encoder Representation from Transformers</h3><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/bert架构.png" alt="Bert架构"></p><p>Bert 是一个预训练的语言表征模型。其主要用到了transformer 的Encoder 模块，在预训练时采用了Masked Language Model（MLM）和Next Sentence Prediction（NSP）方法，强调双向获取信息，充分利用上下文信息，提高模型的理解能力。下面我将就Bert的一些改进进行介绍。</p><h4 id="Encoder-only"><a href="#Encoder-only" class="headerlink" title="Encoder only"></a>Encoder only</h4><p>Bert作为一个特征提取器，其主要目的是进行自监督的预训练学习，学习文本的深层语义信息。因此它仅保留了transformer中的Encoder部分，并且借助Encoder不掩盖后面位置的信息的特点，Bert实现了让模型学习双向的文本信息，结合上下文进行预测。</p><h4 id="Masked-Language-Model"><a href="#Masked-Language-Model" class="headerlink" title="Masked Language Model"></a>Masked Language Model</h4><p>MLM是 BERT 模型的预训练任务之一。在 MLM 任务中，输入文本的一部分词汇会被随机遮盖，模型的任务是根据上下文来预测这些被遮盖的词汇。首先，它随机地将输入文本中的15%的词替换为一个特殊的[MASK]标记，表示这些词被遮盖了。然后，它使用一个基于Transformer的编码器来处理这个遮盖后的文本，得到每个位置的隐藏状态向量。最后，将每个位置的隐藏状态向量输入一个softmax来得到词汇表中所有词的概率分布，进而预测被覆盖的词。MLM任务使得Bert不是像之前的语言模型那样只能从左到右或者从右到左地处理文本。这对于很多自然语言处理的下游任务，如机器阅读理解，自然语言推理，问答系统等，都有很大的帮助。</p><h4 id="Next-Sentence-Prediction"><a href="#Next-Sentence-Prediction" class="headerlink" title="Next Sentence Prediction"></a>Next Sentence Prediction</h4><p>NSP也是BERT 模型的预训练任务之一。在NSP任务中，模型会输入两个句子，模型需要判断这两个句子是否具有上下文关联关系。首先，它从文本语料库中随机选择两个句子作为输入，其中第一个句子称为A，第二个句子称为B。然后，它在两个句子之间加入一个特殊的分隔符[SEP]，并在句子A的开头加入一个特殊的token [CLS]，作为整个输入序列的第一个词。接着，它使用一个基于Transformer的编码器来处理这个输入序列，得到每个位置的隐藏状态向量。最后，它使用一个二分类器来预测句子B是否在原始文档中是句子A的下一句。在训练期间，输入的句子对50％ 在原始文档中是前后关系，另外 50％ 是从语料库中随机组成的无关句子对。</p><h4 id="Improve-of-input-embeddings"><a href="#Improve-of-input-embeddings" class="headerlink" title="Improve of input embeddings"></a>Improve of input embeddings</h4><p>Bert的input embeddings相较于Transformer进行了改进，它将Position Embeddings不再由公式进行转换，而是让模型去学习如何对位置进行编码。并且Bert加入了Segment Embeddings，用来表示两个不同的句子，以匹配它的预训练任务。Bert还开创性的在input embeddings中加入了一些特殊token。如[CLS]，[SEP]，[CLS]经过自注意力机制后的输出就是模型对一整句话的注意力，即模型对句子的特征提取。因为注意力机制的特殊性，把它放在句子中的任意位置效果都相同。而[SEP]则是用来分隔不同的句子的。这些特殊字符在模型中有特殊的含义，后续的ViT模型也借鉴了这种用特殊字符的方法来提高模型的理解能力。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/输入嵌入.png" alt="The input embeddings in Bert"></p><h4 id="Fine-tuning（微调）"><a href="#Fine-tuning（微调）" class="headerlink" title="Fine-tuning（微调）"></a>Fine-tuning（微调）</h4><p>前面提到，Bert本质是一个预训练的特征提取器。它将输入的句子，通过transformer的Encoder进行编码，将句子的特征提取出来。为了将这些句子特征运用到各种自然语言处理任务上，还需要对模型进行微调。比如选择特定的目标函数，将句子的哪些部分的特征作为下游任务的输入等等。在【Bert框架图】中，就具体展示了Bert如何在SQuAD数据集上进行问答任务。</p><h4 id="The-contribution-of-Bert"><a href="#The-contribution-of-Bert" class="headerlink" title="The contribution of Bert"></a>The contribution of Bert</h4><p>Bert模型的贡献在于它引入了预训练和微调体系和开创了双向语义理解的预训练任务。Bert模型只需要在大量无标注的数据上进行无监督训练，然后进行模型迁移和有监督微调，便可应用在不同的任务上。这大大减少了对特定任务的模型训练成本和大量有标注数据的需求。而双向语义理解的预训练任务则很好的提高了模型对上下文结合的理解能力。</p><h3 id="GPT-Generative-Pre-trained-Transformer"><a href="#GPT-Generative-Pre-trained-Transformer" class="headerlink" title="GPT: Generative Pre-trained Transformer"></a>GPT: Generative Pre-trained Transformer</h3><p>GPT是一种基于Transformer的生成式预训练语言模型，它可以根据给定的文本生成连贯和自然的文本，并且可以用于多种自然语言处理的任务，如机器翻译，文本摘要，问答系统等。GPT的核心思想是利用大量的无标注文本数据进行无监督的预训练，学习文本的通用表示，然后在特定的任务上进行有监督的微调，调整模型参数，提高模型性能。GPT有多个版本，如GPT-1，GPT-2，GPT-3等，每个版本都增加了模型的参数数量和训练数据的规模，从而提高了模型的生成能力和泛化能力。下面我将就GPT-1到GPT-3的一些研究论文，介绍一下GPT的特点。</p><h4 id="Decoder-only"><a href="#Decoder-only" class="headerlink" title="Decoder only"></a>Decoder only</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/gpt架构.png" alt=""></p><p>GPT只使用了Transformer的Decoder部分，而没有使用Encoder部分。这是因为GPT的目标是做语言建模，即根据上文预测下一个单词。而语言建模只需要关注之前生成的文本，而不需要关注整个输入序列。因此，GPT只需要使用掩码自注意力层来实现这个功能，而不需要使用编码器-解码器注意力层。另外，GPT还去掉了Decoder中原本用来接收Encoder输出的多头自注意力层，因为GPT没有Encoder部分。GPT和Transformer Decoder的结构对比如图。</p><h4 id="GPT-2-can-be-a-zero-shot-learning-language-model"><a href="#GPT-2-can-be-a-zero-shot-learning-language-model" class="headerlink" title="GPT-2 can be a zero-shot learning language model"></a>GPT-2 can be a zero-shot learning language model</h4><p>在模型大小和训练参数量不断扩大的情况下，GPT-2做到了zero-shot learning（零样本学习）。它可以在没有任何标注数据或者参数调整的情况下，直接执行下游任务。这一点与BERT等模型不同，它们需要在预训练之后进行有监督的微调才能适应特定的任务。当GPT-2要执行一个zero-shot的下游任务时，它只需要根据任务的描述和示例来生成相应的文本。比如，如果要做机器翻译任务，你只需要输入“Translate from English to French: Hello, how are you?”，它就会输出“Bonjour, comment allez-vous?”。如果要做问答任务，你只需要输入“Who is the president of U.S.?”，它就会输出“Joseph Robinette Biden Jr.”。这些输入都可以看作是给模型的提示或者指令（prompt），让模型知道要做什么样的任务，而不需要再去给一些有标注数据让模型进行迁移学习（微调），提高了模型的性能和泛化能力。这是因为GPT-2使用了海量的无标注文本数据进行预训练，学习了通用的语言知识和规律。这些数据包含了各种任务相关的信息，比如语法，逻辑，常识等，让模型可以理解输入的自然语言的语义。GPT-2进而通过自回归的方式，从左到右地预测下一个单词，从而进行语言生成和输出。</p><h4 id="A-particularly-large-GPT-3-exhibits-emergence"><a href="#A-particularly-large-GPT-3-exhibits-emergence" class="headerlink" title="A particularly large GPT-3 exhibits emergence"></a>A particularly large GPT-3 exhibits emergence</h4><p>在GPT-2之后，OpenAI进一步的增加模型大小和训练参数量，最后在一种量变导致质变的情况下，模型的性能取得了质的飞跃，出现了Grokking（”涌现“）。例如模型的准确率和泛化性大大提高，可以通过少量的示例来完成各种下游任务。出现“涌现”的原因尚不可知，人们猜测是因为大模型具有更强的记忆和推理能力，能够从海量的数据中学习到更多的知识和规律。总之在该论文中，GPT-3在参数量提升到1750亿的情况下，模型的能力刷新了很多当时NLP领域的记录。并且论文指出，GPT模型还未见到增加参数量带来的性能饱和现象，这说明继续增加参数量和原始数据，GPT的性能还能进一步的提升。</p><h4 id="Where-to-get-so-much-text-data"><a href="#Where-to-get-so-much-text-data" class="headerlink" title="Where to get so much text data"></a>Where to get so much text data</h4><p>GPT需要大量的文本数据用来学习，如何获得高质量的文本数据也是一个值得研究的方向。OpenAI在GPT-2中使用了一个国外论坛Reddit的数据，并借助Reddit的点赞机制，筛选优质数据。而GPT-3训练所用的数据量又翻了1125倍，来到了45TB，相当于互联网上所有文本的10%。GPT-3借助Common Crawl从互联网爬取的超大数据集，并将Common Crawl进行筛选过滤。其过滤算法是借助GPT-2已筛选好的优质文本作为正例训练LR分类器，用来筛选数据。再通过去重等方法得到优质数据集。</p><h3 id="The-Advantages-of-Pre-trained-Transformer-in-NLP"><a href="#The-Advantages-of-Pre-trained-Transformer-in-NLP" class="headerlink" title="The Advantages of Pre-trained Transformer in NLP"></a>The Advantages of Pre-trained Transformer in NLP</h3><p>由上文可知Transformer预训练模型可以充分利用大规模的无标注文本数据，通过自监督学习的方式，学习到通用的语言知识和表示，从而提高下游任务的性能和泛化能力。并且Transformer预训练模型具有很好的可扩展性和灵活性，可以根据不同的任务需求，调整模型的结构和参数，实现多种功能和应用。Transformer预训练模型是目前NLP领域最先进和最流行的技术之一，它已经在机器翻译、文本分类、阅读理解、对话生成、文本摘要等多个任务上取得了显著的效果，展现了巨大的潜力和前景。</p><h2 id="Transformer-in-CV"><a href="#Transformer-in-CV" class="headerlink" title="Transformer in CV"></a>Transformer in CV</h2><p>在CV领域，transformer也在大放异彩。一开始，人们将transformer中的attention与CNN结合，将attention用于CNN中的特征提取。后来Vision Transformer这篇论文提出，用纯的transformer结构进行预训练，训练出来的模型同样具有很好的效果。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/ViT架构.png" alt="ViT架构"></p><h3 id="How-to-enter-the-image-into-the-Transformer"><a href="#How-to-enter-the-image-into-the-Transformer" class="headerlink" title="How to enter the image into the Transformer"></a>How to enter the image into the Transformer</h3><p>如何将二维的图片转化为一维的序列输入transformer是想要在CV中利用transformer的第一步。这篇论文展示了一种巧妙的方法。即将图片分为一个个patch，将patchs排成一维序列输入transformer便达到了类似NLP中输入一句话的效果。这也是论文表示为什么说AN IMAGE IS WORTH 16X16 WORDS 的原因。至于为什么不将图片的每个像素分割拉直成一维，那是以为这样处理后的序列将会变得很长，增加了模型训练的复杂度。</p><h3 id="Patch-and-Position-Embedding"><a href="#Patch-and-Position-Embedding" class="headerlink" title="Patch and Position Embedding"></a>Patch and Position Embedding</h3><p>将图片分割成一个个patch之后势必会损失图片的某些位置信息。不过好在transformer模型本身就提供了一个Position Embedding，这使得Patch Embedding后的向量可以再加上一个Position Embedding来表示该patch在图片中的位置信息。ViT模型主要借鉴的是前面所说的Bert模型，它在Position Embedding的具体转换方法上同样交给模型去自主学习。</p><h3 id="Extra-learnable-class-embedding"><a href="#Extra-learnable-class-embedding" class="headerlink" title="Extra learnable [class] embedding"></a>Extra learnable [class] embedding</h3><p>在ViT中作者加入了一个额外的可学习的嵌入向量，它用于表示整个图像的类别信息，即ViT架构图中的“0*”。它的目的是从其他图像块的嵌入向量中学习到全局的特征，作为图像的表示向量。这类似Bert模型中的[CLS]token，都是用于获取整个序列的信息。</p><h3 id="Compared-with-CNN"><a href="#Compared-with-CNN" class="headerlink" title="Compared with CNN"></a>Compared with CNN</h3><p>相比于CNN，ViT有一些常见的优点，例如利用无标注数据的能力，更好的并行性，泛化能力更强，鲁棒性更好。这些优点有些也来自transformer模型，在前文中已提及，这里就不再展开。而ViT也不仅仅都是优点，当训练数据集不够大的时候，ViT的表现通常比同等大小的ResNets要差一些，这是因为CNN在CV领域已经得到了成熟的应用，利用transformer难以使用CNN的一些先验知识，如locality/two-dimensional neighborhood structure（图片的二维关系）和translation equivariance（平移不变性）。但当数据集不断扩大时，ViT在多项任务上的性能反超了ResNets。在论文中，作者也提到了将ResNets与transformer结合进行训练（Hybrid），即在图片Embedding时借助ResNets，将图片转化为向量。Hybrid在小数据集上的表现均好于ViT和ResNets，但在大数据集上与却差于ViT。这说明Transformer在CV领域也有取代CNN的势头。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/模型能力对比图.png" alt="模型性能比较"></p><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>在阅读了大量的关于transformer论文后，我们不难发现transformer在广度上不断扩展，从提出之时的机器翻译到NLP整个领域再到CV等其他深度学习的领域，transformer被应用在各种不同的任务中，或与传统RNN和CNN结合，或直接开辟一条新的方法，为不同领域不同任务的解决提供了更多的选择。而在深度上，从时间线上来看，transformer模型也在不断的改善和调整来适应不同的任务。包括在数据量和模型参数量上的不断扩大，将预训练和微调的思想加入transformer，这些研究都在不断的提高transformer模型的学习能力和泛化能力。</p><p>下面我列举一下transformer模型目前在哪些领域已取得了比较大的成功</p><ul><li>自然语言处理： Transformer 及其变体已在 NLP 任务中得到广泛探索和应用，例如机器翻译、语言建模和命名实体识别。大量的努力致力于在大规模文本语料库上预训练 Transformer 模型，这是 Transformer 在 NLP 中广泛应用的主要原因之一。</li><li>计算机视觉： Transformer 还适用于各种视觉任务，例如图像分类、物体检测、图像生成和视频处理。</li><li>音频应用： Transformer 还可以扩展到与音频相关的应用，例如语音识别，语音合成，语音增强和音乐生成 。</li><li>多模态应用：由于其灵活的架构，Transformer 还被应用于各种多模态场景，例如视觉问答、视觉常识推理、字幕生成、语音到文本翻译和文本到图像生成。</li></ul><h3 id="个人思考"><a href="#个人思考" class="headerlink" title="个人思考"></a>个人思考</h3><p>目前关于transformer的研究的应用仍在不断发展。例如在CV领域的swim transformer模型基于ViT模型进行改进，它的主要特点是使用滑动窗口机制，将图像分成多个小块，然后在每个小块内进行自注意力计算，这样可以降低计算复杂度，同时也保留了局部信息。除此之外，Transformer打破NLP和CV研究的鸿沟，为多模态的应用带来了希望。如今我们已经可以用上很多基于Transformer的多模态的应用，例如通过语言描述生成代码、图片、视频、音乐等等，这些应用极大的促进了AIGC，AGI的发展。</p><p>因此在未来的研究中，我们可以进一步改进Transformer，如对Transformer优秀的性能进行理论分析；对Transformer的学习和生成过程进行解释，减少黑箱性；对Transformer的模内和跨模态注意力的设计进行改进。</p><p>相信在众多的研究者的努力下，Transformer的能力可以得到进一步的发展，Transformer可以被运用到更多的领域上，更好的改善我们的生活。</p><h2 id="参考文献"><a href="#参考文献" class="headerlink" title="参考文献"></a>参考文献</h2><ul><li>[1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” <em>Advances in neural information processing systems</em>, vol.30, 2017.</li><li>[2] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” <em>arXiv preprint arXiv:1810.04805</em>, 2018.</li><li>[3] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly <em>et al.</em>, “An image is worth 16x16 words: Transformers for image recognition at scale,” <em>arXiv preprint arXiv:2010.11929</em>, 2020.</li><li>[4] A. Radford, K. Narasimhan, T. Salimans, I. Sutskever <em>et al.</em>, “Improving language understanding by generative pre-training,” 2018.</li><li>[5] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever <em>et al.</em>, “Language models are unsupervised multitask learners,” <em>OpenAI blog</em>, vol.1, no.8, p.9, 2019.</li><li>[6] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell <em>et al.</em>, “Language models are few-shot learners,” <em>Advances in neural information processing systems</em>, vol.33, pp.1877–1901, 2020.</li><li>[7] B. McCann, N. S. Keskar, C. Xiong, and R. Socher, “The natural language decathlon: Multitask learning as question answering,” <em>arXiv preprint arXiv:1806.08730</em>, 2018.</li><li>[8] N. Parmar, A. Vaswani, J. Uszkoreit, L. Kaiser, N. Shazeer, A. Ku, and D. Tran, “Image transformer,” <em>International conference on machine learning</em>. PMLR, pp.4055–4064, 2018.</li><li>[9] T. Lin, Y. Wang, X. Liu, and X. Qiu, “A survey of transformers,” <em>AI Open</em>,2022.</li><li>[10] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” <em>Proceedings of the IEEE/CVF international conference on computer vision</em>, 2021, pp.10012–10022.</li><li>[11] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark <em>et al.</em>, “Learning transferable visual models from natural language supervision,” <em>International conference on machine learning</em>. PMLR, pp. 8748–8763, 2021.</li><li>[12] H. Chefer, S. Gur, and L. Wolf, “Transformer interpretability beyond attention visualization,” <em>Proceedings of the IEEE/CVF conference on computer vision and pattern recognition</em>, pp.782–791, 2021.</li></ul>]]></content>
<categories>
<category> 科研日记 </category>
</categories>
<tags>
<tag> 人工智能 </tag>
<tag> Transformer </tag>
<tag> 综述 </tag>
</tags>
</entry>
<entry>
<title>hexo与Typora插入图片的解决方法</title>
<link href="/2024/02/14/hexo%E4%B8%8ETypora%E6%8F%92%E5%85%A5%E5%9B%BE%E7%89%87%E7%9A%84%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/"/>
<url>/2024/02/14/hexo%E4%B8%8ETypora%E6%8F%92%E5%85%A5%E5%9B%BE%E7%89%87%E7%9A%84%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/</url>
<content type="html"><![CDATA[<h1 id="问题描述"><a href="#问题描述" class="headerlink" title="问题描述"></a>问题描述</h1><p>众所周知，在md文件中插入图片的语法为 <code>![]()</code>。</p><p>其中<strong>方括号</strong>是图片描述，<strong>圆括号</strong>是图片路径。</p><p>一般来说有三种图片路径，分别是<strong>相对路径，绝对路径和网络路径</strong>。</p><h2 id="网络路径（图床）"><a href="#网络路径（图床）" class="headerlink" title="网络路径（图床）"></a>网络路径（图床）</h2><p>所谓的网络路径就是直接引用网上的图片，直接复制图片的网络地址，放在圆括号中就完事了。</p><p>这种方式十分的方便，但是也存在一定的问题：</p><ul><li>图片失效导致无法加载；</li><li>打开网页后要再请求加载图片；</li><li>原网站限制，如微信公众号的图片会变得不可见等。</li></ul><h2 id="绝对路径"><a href="#绝对路径" class="headerlink" title="绝对路径"></a>绝对路径</h2><p>绝对路径是图片在计算机中的绝对位置。</p><h2 id="相对路径"><a href="#相对路径" class="headerlink" title="相对路径"></a>相对路径</h2><p>相对路径是相对于当前文件的路径。</p><p>当采用相对路径时，如果你的Hexo项目中只有少量图片，那最简单的方法就是将它们放在 <code>/source/images</code> 文件夹中。然后通过类似于 <code></code> 的方法访问它们。</p><p>对于那些想要更有规律地提供图片和其他资源以及想要将他们的资源分布在各个文章上的人来说，Hexo也提供了更组织化的方式来管理资源。你需要将 <code>_config.yml</code> 文件中的 <code>post_asset_folder</code> 选项设为 <code>true</code> 来打开<strong>文章资源文件夹</strong>。</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">post_asset_folder: true</span><br></pre></td></tr></table></figure><p>当资源文件管理功能打开后，Hexo将会在你每一次通过 <code>hexo new [layout] <title></code> 命令创建新文章时自动创建一个文件夹在 <code>/source/_posts</code>文件夹中。这个资源文件夹将会有与这个文章文件一样的名字。将所有与你的文章有关的资源放在这个关联文件夹中之后，你可以通过相对路径来引用它们，这样你就得到了一个更简单而且方便得多的工作流。</p><p>例如新建一个博客名为 <code>1.md</code>，相应的会生成一个 <code>1</code>文件夹，在该文件夹中存放图片 <code>image.jpg</code>。<br>在Typora编辑器中打开 <code>1.md</code>，使用 <code></code>能在编辑器中正常引用该图片。<br>在hexo网页中，却无法使用 <code>![]()</code>的相对路径的方法进行引用，本人尝试了很多种相对路径的写法都无法实现。<br>原因可能是 <code>_posts</code>文件夹中的文件在构建html时，其中的图片地址会发生改变，如果图片放在 <code>/source/其他文件夹</code>中，图片正常显示。</p><h1 id="解决方法"><a href="#解决方法" class="headerlink" title="解决方法"></a>解决方法</h1><p>此时如果要插入该图片，可以用以下几种方法：</p><h2 id="使用相对路径引用的标签插件"><a href="#使用相对路径引用的标签插件" class="headerlink" title="使用相对路径引用的标签插件"></a>使用相对路径引用的标签插件</h2><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">{% asset_path slug %}</span><br><span class="line">{% asset_img slug [title] %}</span><br><span class="line">{% asset_link slug [title] %}</span><br></pre></td></tr></table></figure><p>正确的引用该图片方式是使用下列的标签插件：</p><p><code>{% asset_img image.jpg This is an image %}</code></p><h2 id="使用-hexo-asset-image-hexo-asset-img插件"><a href="#使用-hexo-asset-image-hexo-asset-img插件" class="headerlink" title="使用 hexo-asset-image/hexo-asset-img插件"></a>使用 <code>hexo-asset-image</code>/<code>hexo-asset-img</code>插件</h2><p>（较为复杂，本人尝试未成功，可自行搜索）</p><h2 id="使用hexo-renderer-marked插件（推荐）"><a href="#使用hexo-renderer-marked插件（推荐）" class="headerlink" title="使用hexo-renderer-marked插件（推荐）"></a>使用<a href="https://github.com/hexojs/hexo-renderer-marked"><code>hexo-renderer-marked</code></a>插件（推荐）</h2><h3 id="安装方法"><a href="#安装方法" class="headerlink" title="安装方法"></a>安装方法</h3><p>用 <code>npm install hexo-renderer-marked</code>命令安装（新版hexo已集成，无需安装）</p><h3 id="使用方法"><a href="#使用方法" class="headerlink" title="使用方法"></a>使用方法</h3><p>在 <code>_config.yml</code>中更改配置如下：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">post_asset_folder: true</span><br><span class="line">marked:</span><br><span class="line"> prependRoot: true</span><br><span class="line"> postAsset: true</span><br></pre></td></tr></table></figure><p>第一行在前文已修改，后三行需要自行添加到第一行后。<br>此时在hexo网页中，使用 <code></code>的相对路径的方法可成功进行引用。<br>（注意：路径中不能有空格）</p><h1 id="方法完善"><a href="#方法完善" class="headerlink" title="方法完善"></a>方法完善</h1><p>注意到，如果使用 <code></code>的相对路径在typora本地无法正确引用图片（typora需要使用 <code></code>）<br>在文章的front-matter中添加一行 <code>typora-root-url: 文章标题</code>。那么在typora中所有的路径前都会加上 <code>文章标题/</code>。（<code>image.jpg->1/image.jpg</code>）<br>进一步可以在 <code>scaffolds</code>文件夹中的文章模板中添加 <code>typora-root-url: {{title}}</code>这样每次执行 <code>hexo new</code>命令新建文章的时候，会在front-matter中自动添加该配置。</p><h1 id="hexo与Typora的进一步联动"><a href="#hexo与Typora的进一步联动" class="headerlink" title="hexo与Typora的进一步联动"></a>hexo与Typora的进一步联动</h1><p>文章资源文件夹中引用图片，<strong>前提是先将图片放入到文章资源文件夹</strong>，如果图片数量众多的话，一张一张的放很影响效率。<br>使用Typora可以在复制图片到文章中时，自动将图片复制到指定文件夹，并修改成对应的路径。</p><h2 id="设置方法"><a href="#设置方法" class="headerlink" title="设置方法"></a>设置方法</h2><p>找到Typora的偏好设置->图像，进行如下设置</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/typora设置方法.png" alt=""></p><p><code>./${filename}</code>设置与hexo的文章资源文件夹设置相对应，<strong>至此写博客时插入图片只需将图片复制（拖动）到文章中即可。</strong></p><h1 id="实际工作流"><a href="#实际工作流" class="headerlink" title="实际工作流"></a>实际工作流</h1><p>方法一：使用 <code>hexo new <layout> "filename"</code>指令生成md文件（同时生成资源文件夹），然后使用typora打开md文件进行书写。</p><p>方法二：在 <code>/source/_posts</code>新建md文件，然后使用typora打开md文件进行书写，在插入图片时会自动生成资源文件夹。</p>]]></content>
<categories>
<category> 经验之谈 </category>
</categories>
<tags>
<tag> 博客 </tag>
<tag> hexo </tag>
<tag> Typora </tag>
<tag> markdown </tag>
</tags>
</entry>
<entry>
<title>Git教程</title>
<link href="/2024/02/14/Git%E6%95%99%E7%A8%8B/"/>
<url>/2024/02/14/Git%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<h1 id="Git-分布式版本控制工具"><a href="#Git-分布式版本控制工具" class="headerlink" title="Git - 分布式版本控制工具"></a>Git - 分布式版本控制工具</h1><p>教程视频：<a href="https://www.bilibili.com/video/BV1MU4y1Y7h5">https://www.bilibili.com/video/BV1MU4y1Y7h5</a></p><h2 id="1、目标"><a href="#1、目标" class="headerlink" title="1、目标"></a>1、目标</h2><ul><li>了解Git基本概念</li><li>能够概述Git工作流程</li><li>能够使用Git常用命令</li><li>熟悉Git代码托管服务</li><li>能够使用IDEA操作Git</li></ul><h2 id="2、概述"><a href="#2、概述" class="headerlink" title="2、概述"></a>2、概述</h2><h3 id="2-1、开发中的实际场景"><a href="#2-1、开发中的实际场景" class="headerlink" title="2.1、开发中的实际场景"></a>2.1、开发中的实际场景</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">场景一：备份</span><br><span class="line">小明负责的模块就要完成了，就在即将Release之前的一瞬间，电脑突然蓝屏，硬盘光荣牺牲！几个月来的努力付之东流</span><br><span class="line"></span><br><span class="line">场景二：代码还原</span><br><span class="line">这个项目中需要一个很复杂的功能，老王摸索了一个星期终于有眉目了，可是这被改得面目全非的代码已经回不到从前了。什么地方能买到哆啦A梦的时光机啊？</span><br><span class="line"></span><br><span class="line">场景三：协同开发</span><br><span class="line">小刚和小强先后从文件服务器上下载了同一个文件：Analysis.java。小刚在Analysis.java文件中的第30行声明了一个方法，叫count()，先保存到了文件服务器上；小强在Analysis.java文件中的第50行声明了一个方法，叫sum()，也随后保存到了文件服务器上，于是，count()方法就只存在于小刚的记忆中了</span><br><span class="line"></span><br><span class="line">场景四：追溯问题代码的编写人和编写时间！</span><br><span class="line">老王是另一位项目经理，每次因为项目进度挨骂之后，他都不知道该扣哪个程序员的工资！就拿这次来说吧，有个Bug调试了30多个小时才知道是因为相关属性没有在应用初始化时赋值！可是二胖、王东、刘流和正经牛都不承认是自己干的！</span><br></pre></td></tr></table></figure><h3 id="2-2、版本控制器的方式"><a href="#2-2、版本控制器的方式" class="headerlink" title="2.2、版本控制器的方式"></a>2.2、版本控制器的方式</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">a、集中式版本控制工具</span><br><span class="line">集中式版本控制工具，版本库是集中存放在中央服务器的，team里每个人work时从中央服务器下载代码，是必须联网才能工作，局域网或互联网。个人修改后然后提交到中央版本库。</span><br><span class="line">举例：SVN和CVS</span><br><span class="line">b、分布式版本控制工具</span><br><span class="line">分布式版本控制系统没有“中央服务器”，每个人的电脑上都是一个完整的版本库，这样工作的时候，无需要联网了，因为版本库就在你自己的电脑上。多人协作只需要各自的修改推送给对方，就能互相看到对方的修改了。</span><br><span class="line">举例：Git</span><br></pre></td></tr></table></figure><h3 id="2-3、SVN"><a href="#2-3、SVN" class="headerlink" title="2.3、SVN"></a>2.3、SVN</h3><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/SVN.png" alt="SVN"></p><h3 id="2-4、Git"><a href="#2-4、Git" class="headerlink" title="2.4、Git"></a>2.4、Git</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">Git是分布式的,Git不需要有中心服务器，我们每台电脑拥有的东西都是一样的。我们使用Git并且有个中心服务器，仅仅是为了方便交换大家的修改，但是这个服务器的地位和我们每个人的PC是一样的。我们可以把它当做一个开发者的pc就可以就是为了大家代码容易交流不关机用的。没有它大家一样可以工作，只不过“交换”修改不方便而已。</span><br><span class="line">git是一个开源的分布式版本控制系统，可以有效、高速地处理从很小到非常大的项目版本管理。Git是Linus Torvalds 为了帮助管理 Linux 内核开发而开发的一个开放源码的版本控制软件。</span><br><span class="line">同生活中的许多伟大事物一样，Git 诞生于一个极富纷争大举创新的年代。Linux 内核开源项目有着为数众多的参与者。 绝大多数的 Linux 内核维护工作都花在了提交补丁和保存归档的繁琐事务上（1991－2002年间）。 到 2002 年，整个项目组开始启用一个专有的分布式版本控制系统 BitKeeper 来管理和维护代码。</span><br><span class="line">到了 2005 年，开发 BitKeeper 的商业公司同 Linux 内核开源社区的合作关系结束，他们收回了Linux 内核社区免费使用 BitKeeper 的权力。 这就迫使 Linux 开源社区（特别是 Linux 的缔造者Linus Torvalds）基于使用 BitKeeper 时的经验教训，开发出自己的版本系统。 他们对新的系统制订了若干目标：</span><br><span class="line"> 速度</span><br><span class="line"> 简单的设计</span><br><span class="line"> 对非线性开发模式的强力支持（允许成千上万个并行开发的分支）</span><br><span class="line"> 完全分布式</span><br><span class="line"> 有能力高效管理类似 Linux 内核一样的超大规模项目（速度和数据量）</span><br></pre></td></tr></table></figure><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/Git.png" alt="Git"></p><h3 id="2-5、Git工作流程图"><a href="#2-5、Git工作流程图" class="headerlink" title="2.5、Git工作流程图"></a>2.5、Git工作流程图</h3><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/Git%E5%B7%A5%E4%BD%9C%E6%B5%81%E7%A8%8B%E5%9B%BE.png" alt="Git工作流程图"></p><p>命令如下：</p><ol><li>clone（克隆）: 从远程仓库中克隆代码到本地仓库</li><li>checkout （检出）:从本地仓库中检出一个仓库分支然后进行修订</li><li>add（添加）: 在提交前先将代码提交到暂存区</li><li>commit（提交）: 提交到本地仓库。本地仓库中保存修改的各个历史版本</li><li>fetch (抓取) ： 从远程库，抓取到本地仓库，不进行任何的合并动作，一般操作比较少。</li><li>pull (拉取) ： 从远程库拉到本地库，自动进行合并(merge)，然后放到到工作区，相当于fetch+merge</li><li>push（推送） : 修改完成后，需要和团队成员共享代码时，将代码推送到远程仓库</li></ol><h2 id="3、Git安装与常用命令"><a href="#3、Git安装与常用命令" class="headerlink" title="3、Git安装与常用命令"></a>3、Git安装与常用命令</h2><p>本教程里的git命令例子都是在Git Bash中演示的，会用到一些基本的linux命令，在此为大家提前列举：</p><ul><li><code>ls</code>/<code>ll</code> 查看当前目录</li><li><code>cat</code> 查看文件内容</li><li><code>touch</code> 创建文件</li><li><code>vi</code> vi编辑器（使用vi编辑器是为了方便展示效果，学员可以记事本、EditPlus、NotePad++等其它编辑器）</li></ul><h3 id="3-1、-Git环境配置"><a href="#3-1、-Git环境配置" class="headerlink" title="3.1、 Git环境配置"></a>3.1、 Git环境配置</h3><h4 id="3-1-1-下载与安装"><a href="#3-1-1-下载与安装" class="headerlink" title="3.1.1 下载与安装"></a>3.1.1 下载与安装</h4><p>下载地址： <a href="https://git-scm.com/download">https://git-scm.com/download</a></p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/Git%E4%B8%8B%E8%BD%BD%E4%B8%8E%E5%AE%89%E8%A3%85.png" alt="Git下载与安装"></p><p>下载完成后可以得到如下安装文件：</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/Git安装包.png" alt="Git安装包"></p><p>双击下载的安装文件来安装Git。安装完成后在电脑桌面（也可以是其他目录)点击右键，如果能够看到如下两个菜单则说明Git安装成功。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/Git%E5%8F%B3%E9%94%AE%E8%8F%9C%E5%8D%95.png" alt="Git右键菜单"></p><p>备注：</p><p>Git GUI：Git提供的图形界面工具</p><p>Git Bash：Git提供的命令行工具</p><p>当安装Git后首先要做的事情是设置用户名称和email地址。这是非常重要的，因为每次Git提交都会使用该用户信息</p><h4 id="3-1-2基本配置"><a href="#3-1-2基本配置" class="headerlink" title="3.1.2基本配置"></a>3.1.2基本配置</h4><ol><li>打开Git Bash</li><li>设置用户信息<br><code>git config --global user.name “itcast”</code><br><code>git config --global user.email “hello@itcast.cn”</code></li></ol><p>查看配置信息<br><code>git config --global user.name</code><br><code>git config --global user.email</code></p><h4 id="3-1-3-为常用指令配置别名（可选）"><a href="#3-1-3-为常用指令配置别名（可选）" class="headerlink" title="3.1.3 为常用指令配置别名（可选）"></a>3.1.3 为常用指令配置别名（可选）</h4><p>有些常用的指令参数非常多，每次都要输入好多参数，我们可以使用别名。</p><ol><li>打开用户目录，创建 <code>.bashrc</code>文件 <code>touch .bashrc</code><br>部分windows系统不允许用户创建点号开头的文件，可以打开gitBash,执行 <code>touch ~/.bashrc</code><br><code><img src="创建bashrc.png" alt="创建bashrc" /></code></li><li><p>在 <code>.bashrc</code>文件中输入如下内容：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash">用于输出git提交日志</span></span><br><span class="line">alias git-log='git log --pretty=oneline --all --graph --abbrev-commit'</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash">用于输出当前目录所有文件及基本信息</span></span><br><span class="line">alias ll='ls -al'</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash">用于一次性暂存并提交所有修改和删除的文件</span></span><br><span class="line">alias commit='git commit -a'</span><br></pre></td></tr></table></figure></li><li>打开gitBash，执行 <code>source ~/.bashrc</code><br><code><img src="bashrc.png" alt="bashrc" /></code></li></ol><h4 id="3-1-4-解决GitBash乱码问题"><a href="#3-1-4-解决GitBash乱码问题" class="headerlink" title="3.1.4 解决GitBash乱码问题"></a>3.1.4 解决GitBash乱码问题</h4><ol><li><p>打开GitBash执行下面命令</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git config --global core.quotepath false</span><br></pre></td></tr></table></figure></li><li><p><code>${git_home}/etc/bash.bashrc</code>文件最后加入下面两行</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">export LANG="zh_CN.UTF-8"</span><br><span class="line">export LC_ALL="zh_CN.UTF-8"</span><br></pre></td></tr></table></figure></li></ol><h3 id="3-2、获取本地仓库"><a href="#3-2、获取本地仓库" class="headerlink" title="3.2、获取本地仓库"></a>3.2、获取本地仓库</h3><p>要使用Git对我们的代码进行版本控制，首先需要获得本地仓库<br>1）在电脑的任意位置创建一个空目录（例如test）作为我们的本地Git仓库<br>2）进入这个目录中，点击右键打开Git bash窗口<br>3）执行命令 <code>git init</code><br>4）如果创建成功后可在文件夹下看到隐藏的.git目录。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E8%8E%B7%E5%8F%96%E6%9C%AC%E5%9C%B0%E4%BB%93%E5%BA%93.png" alt="获取本地仓库"></p><h3 id="3-3、基础操作指令"><a href="#3-3、基础操作指令" class="headerlink" title="3.3、基础操作指令"></a>3.3、基础操作指令</h3><p>Git工作目录下对于文件的<strong>修改</strong>（增加、删除、更新）会存在几个状态，这些修改的状态会随着我们执行Git的命令而发生变化。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%9F%BA%E7%A1%80%E6%93%8D%E4%BD%9C%E6%8C%87%E4%BB%A4.png" alt="基础操作指令"></p><p>本章节主要讲解如何使用命令来控制这些状态之间的转换：</p><ol><li>git add (工作区 —> 暂存区)</li><li>git commit (暂存区 —> 本地仓库)</li></ol><h4 id="3-3-1、-查看修改的状态-status"><a href="#3-3-1、-查看修改的状态-status" class="headerlink" title="3.3.1、*查看修改的状态(status)"></a>3.3.1、*查看修改的状态(status)</h4><ul><li>作用：查看的修改的状态（暂存区、工作区）</li><li>命令形式：<code>git status</code></li></ul><h4 id="3-3-2、-添加工作区到暂存区-add"><a href="#3-3-2、-添加工作区到暂存区-add" class="headerlink" title="3.3.2、*添加工作区到暂存区(add)"></a>3.3.2、*添加工作区到暂存区(add)</h4><ul><li>作用：添加工作区一个或多个文件的修改到暂存区</li><li>命令形式：<code>git add 单个文件名|通配符</code><ul><li>将所有修改加入暂存区：<code>git add .</code></li></ul></li></ul><h4 id="3-3-3、-提交暂存区到本地仓库-commit"><a href="#3-3-3、-提交暂存区到本地仓库-commit" class="headerlink" title="3.3.3、*提交暂存区到本地仓库(commit)"></a>3.3.3、*提交暂存区到本地仓库(commit)</h4><ul><li>作用：提交暂存区内容到本地仓库的当前分支</li><li>命令形式：<code>git commit -m '注释内容'</code></li></ul><h4 id="commit"><a href="#commit" class="headerlink" title="commit"></a>commit</h4><ul><li>命令形式：<code>git commit [options]</code><ul><li>options<ul><li><code>-a</code> = <code>--all</code><br>automatically stage files that have been <strong>modified and deleted</strong>.<br>（新创建的文件仍需 <code>git add</code>）</li></ul></li></ul></li></ul><h4 id="3-3-4、-查看提交日志-log"><a href="#3-3-4、-查看提交日志-log" class="headerlink" title="3.3.4、*查看提交日志(log)"></a>3.3.4、*查看提交日志(log)</h4><p><strong>在<a href="#3.1.3 为常用指令配置别名（可选）">3.1.3</a>中配置的别名 <code>git-log</code>就包含了这些参数，所以后续可以直接使用指令 <code>git-log</code></strong></p><ul><li>作用:查看提交记录</li><li>命令形式：<code>git log [option]</code><ul><li>options<ul><li><code>--all</code> 显示所有分支</li><li><code>--pretty=oneline</code> 将提交信息显示为一行</li><li><code>--abbrev-commit</code> 使得输出的commitID更简短</li><li><code>--graph</code> 以图的形式显示</li></ul></li></ul></li></ul><h4 id="3-3-5、版本回退"><a href="#3-3-5、版本回退" class="headerlink" title="3.3.5、版本回退"></a>3.3.5、版本回退</h4><ul><li>作用：版本切换</li><li>命令形式：<code>git reset --hard commitID</code><ul><li>commitID 可以使用 <code>git-log</code>或 <code>git log</code>指令查看</li></ul></li><li>如何查看已经删除的记录？<ul><li><code>git reflog</code></li><li>这个指令可以看到已经删除的提交记录</li></ul></li></ul><h4 id="3-3-6、添加文件至忽略列表"><a href="#3-3-6、添加文件至忽略列表" class="headerlink" title="3.3.6、添加文件至忽略列表"></a>3.3.6、添加文件至忽略列表</h4><p>一般我们总会有些文件无需纳入Git 的管理，也不希望它们总出现在未跟踪文件列表。 通常都是些自动生成的文件，比如日志文件，或者编译过程中创建的临时文件等。 在这种情况下，我们可以在工作目录中创建一个名为 <code>.gitignore</code>的文件（文件名称固定），列出要忽略的文件模式。下面是一个示例：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">no .a files</span></span><br><span class="line">*.a</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">but <span class="keyword">do</span> track lib.a, even though you<span class="string">'re ignoring .a files above</span></span></span><br><span class="line">!lib.a</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash"><span class="string">only ignore the '</span>TODO<span class="string">' file in the current directory, not subdir/TODO</span></span></span><br><span class="line">/TODO</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash"><span class="string">ignore all files in the '</span>build/<span class="string">' directory</span></span></span><br><span class="line">build/</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash"><span class="string">ignore doc/notes.txt, but not doc/server/arch.txt</span></span></span><br><span class="line">doc/*.txt</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash"><span class="string">ignore all .pdf files in the doc/ directory</span></span></span><br><span class="line">doc/**/*.pdf</span><br></pre></td></tr></table></figure><h4 id="练习-基础操作"><a href="#练习-基础操作" class="headerlink" title="练习:基础操作"></a>练习:基础操作</h4><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">####################仓库初始化######################</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">创建目录（git_test01）并在目录下打开gitbash</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">初始化git仓库</span></span><br><span class="line">git init</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">####################创建文件并提交#####################</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">目录下创建文件 file01.txt</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">将修改加入暂存区</span></span><br><span class="line">git add .</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">将修改提交到本地仓库，提交记录内容为：commit 001</span></span><br><span class="line">git commit -m 'commit 001'</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">查看日志</span></span><br><span class="line">git log</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">###################修改文件并提交######################</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">修改file01的内容为：count=1</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">将修改加入暂存区</span></span><br><span class="line">git add .</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash"><span class="comment"># 将修改提交到本地仓库，提交记录内容为：update file01</span></span></span><br><span class="line">git commit -m 'update file01'</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">查看日志</span></span><br><span class="line">git log</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">以精简的方式显示提交记录</span></span><br><span class="line">git-log</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">###################将最后一次修改还原##################</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">查看提交记录</span></span><br><span class="line">git-log</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">找到倒数第2次提交的commitID</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">版本回退</span></span><br><span class="line">git reset commitID --hard</span><br></pre></td></tr></table></figure><h3 id="3-4、分支"><a href="#3-4、分支" class="headerlink" title="3.4、分支"></a>3.4、分支</h3><p>几乎所有的版本控制系统都以某种形式支持分支。 使用分支意味着你可以把你的工作从开发主线上分离开来进行重大的Bug修改、开发新的功能，以免影响开发主线。</p><h4 id="3-4-1、查看本地分支"><a href="#3-4-1、查看本地分支" class="headerlink" title="3.4.1、查看本地分支"></a>3.4.1、查看本地分支</h4><ul><li>命令：<code>git branch</code></li></ul><h4 id="3-4-2、创建本地分支"><a href="#3-4-2、创建本地分支" class="headerlink" title="3.4.2、创建本地分支"></a>3.4.2、创建本地分支</h4><ul><li>命令：<code>git branch 分支名</code></li></ul><h4 id="3-4-4、-切换分支-checkout"><a href="#3-4-4、-切换分支-checkout" class="headerlink" title="3.4.4、*切换分支(checkout)"></a>3.4.4、*切换分支(checkout)</h4><ul><li>命令：<code>git checkout 分支名</code></li></ul><p>我们还可以直接切换到一个不存在的分支（创建并切换）</p><ul><li>命令：<code>git checkout -b 分支名</code></li></ul><h4 id="3-4-6、-合并分支-merge"><a href="#3-4-6、-合并分支-merge" class="headerlink" title="3.4.6、*合并分支(merge)"></a>3.4.6、*合并分支(merge)</h4><p>一个分支（合并分支）上的提交可以合并到另一个分支（目标分支）</p><ul><li>需要先切换到目标分支：<code>git checkout 目标分支名</code></li><li>合并命令：<code>git merge 合并分支名</code></li></ul><h4 id="3-4-7、删除分支"><a href="#3-4-7、删除分支" class="headerlink" title="3.4.7、删除分支"></a>3.4.7、删除分支</h4><p><strong>不能删除当前分支，只能删除其他分支</strong></p><ul><li><code>git branch -d 分支名</code> 删除分支时，需要做各种检查</li><li><code>git branch -D 分支名</code> 不做任何检查，强制删除</li><li>E.g.<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%88%A0%E9%99%A4%E5%88%86%E6%94%AF.png" alt="删除分支"></li></ul><h4 id="3-4-8、解决冲突"><a href="#3-4-8、解决冲突" class="headerlink" title="3.4.8、解决冲突"></a>3.4.8、解决冲突</h4><p>当两个分支上对文件的修改可能会存在冲突，例如同时修改了同一个文件的同一行，这时就需要手动解决冲突，解决冲突步骤如下：</p><ol><li>处理文件中冲突的地方</li><li>将解决完冲突的文件加入暂存区(add)</li><li>提交到仓库(commit)</li></ol><p>冲突部分的内容处理如下所示：</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%86%B2%E7%AA%81%E5%A4%84%E7%90%86.png" alt="冲突处理.png"></p><h4 id="3-4-9、开发中分支使用原则与流程"><a href="#3-4-9、开发中分支使用原则与流程" class="headerlink" title="3.4.9、开发中分支使用原则与流程"></a>3.4.9、开发中分支使用原则与流程</h4><p>几乎所有的版本控制系统都以某种形式支持分支。 使用分支意味着你可以把你的工作从开发主线上分离开来进行重大的Bug修改、开发新的功能，以免影响开发主线。</p><p>在开发中，一般有如下分支使用原则与流程：</p><ul><li>master （生产） 分支<br>线上分支，主分支，中小规模项目作为线上运行的应用对应的分支。</li><li>develop（开发）分支<br>是从master创建的分支，一般作为开发部门的主要开发分支，如果没有其他并行开发不同期上线要求，都可以在此版本进行开发，阶段开发完成后，需要是合并到master分支,准备上线。</li><li>feature/xxxx分支<br>从develop创建的分支，一般是同期并行开发，但不同期上线时创建的分支，分支上的研发任务完成后合并到develop分支。</li><li>hotfix/xxxx分支，<br>从master派生的分支，一般作为线上bug修复使用，修复完成后需要合并到master、test、develop分支。</li><li>还有一些其他分支，在此不再详述，例如test分支（用于代码测试）、pre分支（预上线分支）等等。</li><li><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%88%86%E6%94%AF.png" alt="分支"></li></ul><h4 id="练习-分支操作"><a href="#练习-分支操作" class="headerlink" title="练习:分支操作"></a>练习:分支操作</h4><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">##########################创建并切换到dev01分支，在dev01分支提交</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]创建分支dev01</span></span><br><span class="line">git branch dev01</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]切换到dev01</span></span><br><span class="line">git checkout dev01</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[dev01]创建文件file02.txt</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[dev01]将修改加入暂存区并提交到仓库,提交记录内容为：add file02 on dev</span></span><br><span class="line">git add .</span><br><span class="line">git commit -m 'add file02 on dev'</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[dev01]以精简的方式显示提交记录</span></span><br><span class="line">git-log</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">##########################切换到master分支，将dev01合并到master分支</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[dev01]切换到master分支</span></span><br><span class="line">git checkout master</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]合并dev01到master分支</span></span><br><span class="line">git merge dev01</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]以精简的方式显示提交记录</span></span><br><span class="line">git-log</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]查看文件变化(目录下也出现了file02.txt)</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">#########################删除dev01分支</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]删除dev01分支</span></span><br><span class="line">git branch -d dev01</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[master]以精简的方式显示提交记录</span></span><br><span class="line">git-log</span><br></pre></td></tr></table></figure><h2 id="4、Git远程仓库"><a href="#4、Git远程仓库" class="headerlink" title="4、Git远程仓库"></a>4、Git远程仓库</h2><h3 id="4-1、-常用的托管服务-远程仓库"><a href="#4-1、-常用的托管服务-远程仓库" class="headerlink" title="4.1、 常用的托管服务[远程仓库]"></a>4.1、 常用的托管服务[远程仓库]</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">前面我们已经知道了Git中存在两种类型的仓库，即本地仓库和远程仓库。那么我们如何搭建Git远程仓库呢？我们可以借助互联网上提供的一些代码托管服务来实现，其中比较常用的有GitHub、码云、GitLab等。</span><br><span class="line">gitHub（ 地址：https://github.com/ ）是一个面向开源及私有软件项目的托管平台，因为只支持Git 作为唯一的版本库格式进行托管，故名gitHub</span><br><span class="line">码云（地址： https://gitee.com/ ）是国内的一个代码托管平台，由于服务器在国内，所以相比于GitHub，码云速度会更快</span><br><span class="line">GitLab （地址： https://about.gitlab.com/ ）是一个用于仓库管理系统的开源项目，使用Git作为代码管理工具，并在此基础上搭建起来的web服务,一般用于在企业、学校等内部网络搭建git私服。</span><br></pre></td></tr></table></figure><h3 id="4-2、-注册码云"><a href="#4-2、-注册码云" class="headerlink" title="4.2、 注册码云"></a>4.2、 注册码云</h3><p>要想使用码云的相关服务，需要注册账号（地址： <a href="https://gitee.com/signup">https://gitee.com/signup</a> ）</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E6%B3%A8%E5%86%8C%E7%A0%81%E4%BA%91.png" alt="注册码云"></p><h3 id="4-3、-创建远程仓库"><a href="#4-3、-创建远程仓库" class="headerlink" title="4.3、 创建远程仓库"></a>4.3、 创建远程仓库</h3><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%88%9B%E5%BB%BA%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93.png" alt="创建远程仓库"></p><p>仓库创建完成后可以看到仓库地址，如下图所示:</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E4%BB%93%E5%BA%93%E5%9C%B0%E5%9D%80.png" alt="仓库地址"></p><h3 id="4-4、配置SSH公钥"><a href="#4-4、配置SSH公钥" class="headerlink" title="4.4、配置SSH公钥"></a>4.4、配置SSH公钥</h3><ul><li>生成SSH公钥<ul><li><code>ssh-keygen -t rsa</code></li><li>不断回车<ul><li>如果公钥已经存在，则自动覆盖</li></ul></li></ul></li><li>Gitee设置账户共公钥<ul><li>获取公钥<ul><li><code>cat ~/.ssh/id_rsa.pub</code></li></ul></li><li><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/SSH%E5%85%AC%E9%92%A5.png" alt="SSH公钥"></li><li>验证是否配置成功<ul><li><code>ssh -T git@gitee.com</code></li></ul></li></ul></li></ul><h3 id="4-5、操作远程仓库"><a href="#4-5、操作远程仓库" class="headerlink" title="4.5、操作远程仓库"></a>4.5、操作远程仓库</h3><h4 id="4-5-1、添加远程仓库"><a href="#4-5-1、添加远程仓库" class="headerlink" title="4.5.1、添加远程仓库"></a>4.5.1、添加远程仓库</h4><p><strong>此操作是先初始化本地库，然后与已创建的远程库进行对接。</strong></p><ul><li>命令： <code>git remote add <远端名称> <仓库路径></code><ul><li>远端名称，默认是origin，取决于远端服务器设置</li><li>仓库路径，从远端服务器获取此URL</li><li>例如：<br><code>git remote add origin git@gitee.com:czbk_zhang_meng/git_test.git</code><br><code><img src="添加远程仓库.png" alt="添加远程仓库" /></code></li></ul></li></ul><h4 id="4-5-2、查看远程仓库"><a href="#4-5-2、查看远程仓库" class="headerlink" title="4.5.2、查看远程仓库"></a>4.5.2、查看远程仓库</h4><ul><li>命令：<code>git remote</code><br><code><img src="查看远程仓库.png" alt="/查看远程仓库" /></code></li></ul><h4 id="4-5-3、推送到远程仓库"><a href="#4-5-3、推送到远程仓库" class="headerlink" title="4.5.3、推送到远程仓库"></a>4.5.3、推送到远程仓库</h4><ul><li>命令：<code>git push[ -f][ --set-upstream][ 远端名称[ 本地分支名[:远端分支名]]]</code><ul><li>如果远程分支名和本地分支名称相同，则可以只写本地分支<ul><li><code>git push origin master</code>=<code>git push origin master:master</code><br><img src="/推送到远程仓库-两种写法.png" alt="推送到远程仓库-两种写法"></li></ul></li><li><code>-f</code> = <code>--force</code> 表示强制覆盖</li><li><code>-u</code> = <code>--set-upstream</code> 推送到远端的同时并且建立起和远端分支的关联关系。<ul><li><code>git branch -M master main</code></li><li><code>git push --set-upstream origin main</code></li></ul></li><li>如果<strong>当前分支已经和远端分支关联</strong>，则可以省略分支名和远端名。<ul><li><code>git push</code> 将master分支推送到已关联的远端分支。<br><img src="/推送到远程仓库.png" alt="推送到远程仓库"></li></ul></li></ul></li></ul><p>查询远程仓库</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E6%9F%A5%E8%AF%A2%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93.png" alt="查询远程仓库"></p><h4 id="4-5-4、-本地分支与远程分支的关联关系"><a href="#4-5-4、-本地分支与远程分支的关联关系" class="headerlink" title="4.5.4、 本地分支与远程分支的关联关系"></a>4.5.4、 本地分支与远程分支的关联关系</h4><ul><li>查看关联关系我们可以使用 <code>git branch -vv</code> 命令</li></ul><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E6%9F%A5%E7%9C%8B%E6%9C%AC%E5%9C%B0%E5%88%86%E6%94%AF%E5%92%8C%E8%BF%9C%E7%A8%8B%E5%88%86%E6%94%AF%E7%9A%84%E5%85%B3%E8%81%94%E5%85%B3%E7%B3%BB.png" alt="查看本地分支和远程分支的关联关系"></p><h4 id="4-5-5、从远程仓库克隆"><a href="#4-5-5、从远程仓库克隆" class="headerlink" title="4.5.5、从远程仓库克隆"></a>4.5.5、从远程仓库克隆</h4><p>如果已经有一个远端仓库，我们可以直接clone到本地。</p><ul><li>命令: <code>git clone <仓库路径> [本地目录]</code><ul><li>本地目录可以省略，会自动生成一个目录</li></ul></li></ul><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%85%8B%E9%9A%86%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93%E5%88%B0%E6%9C%AC%E5%9C%B0.png" alt="克隆远程仓库到本地"></p><h4 id="4-5-6、从远程仓库中抓取和拉取"><a href="#4-5-6、从远程仓库中抓取和拉取" class="headerlink" title="4.5.6、从远程仓库中抓取和拉取"></a>4.5.6、从远程仓库中抓取和拉取</h4><p>远程分支和本地的分支一样，我们可以进行merge操作，只是需要先把远端仓库里的更新都下载到本地，再进行操作。</p><ul><li>抓取 命令：<code>git fetch [remote name] [branch name]</code><ul><li><strong>抓取指令就是将仓库里的更新都抓取到本地，不会进行合并</strong></li><li>如果不指定远端名称和分支名，则抓取所有分支。</li></ul></li><li>拉取 命令：<code>git pull [remote name] [branch name]</code><ul><li><strong>拉取指令就是将远端仓库的修改拉到本地并自动进行合并，等同于fetch+merge</strong></li><li>如果不指定远端名称和分支名，则抓取所有并更新当前分支。</li></ul></li></ul><ol><li>在test01这个本地仓库进行一次提交并推送到远程仓库<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%9C%A8test01%E8%BF%99%E4%B8%AA%E6%9C%AC%E5%9C%B0%E4%BB%93%E5%BA%93%E8%BF%9B%E8%A1%8C%E4%B8%80%E6%AC%A1%E6%8F%90%E4%BA%A4%E5%B9%B6%E6%8E%A8%E9%80%81%E5%88%B0%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93.png" alt="在test01这个本地仓库进行一次提交并推送到远程仓库"></li><li>在另一个仓库将远程提交的代码拉取到本地仓库<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%B0%86%E4%BB%93%E5%BA%93%E9%87%8C%E8%BF%9C%E7%A8%8B%E6%8F%90%E4%BA%A4%E7%9A%84%E4%BB%A3%E7%A0%81%E6%8B%89%E5%8F%96%E5%88%B0%E6%9C%AC%E5%9C%B0%E4%BB%93%E5%BA%93.png" alt="将仓库里远程提交的代码拉取到本地仓库"></li></ol><h4 id="4-5-7、解决合并冲突"><a href="#4-5-7、解决合并冲突" class="headerlink" title="4.5.7、解决合并冲突"></a>4.5.7、解决合并冲突</h4><p>在一段时间，A、B用户修改了同一个文件，且修改了同一行位置的代码，此时会发生合并冲突。</p><p>A用户在本地修改代码后优先推送到远程仓库，此时B用户在本地修订代码，提交到本地仓库后，也需要推送到远程仓库，此时B用户晚于A用户，<strong>故需要先拉取远程仓库的提交，经过合并后才能推送到远端分支</strong>，如下图所示。</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E8%A7%A3%E5%86%B3%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93%E5%90%88%E5%B9%B6%E5%86%B2%E7%AA%81.png" alt="解决远程仓库合并冲突"></p><p>在B用户拉取代码时，因为A、B用户同一段时间修改了同一个文件的相同位置代码，故会发生合并冲突。</p><p><strong>远程分支也是分支，所以合并时冲突的解决方式也和解决本地分支冲突相同相同</strong>，在此不再赘述，需要学员自己练习。</p><h4 id="练习-远程仓库操作"><a href="#练习-远程仓库操作" class="headerlink" title="练习:远程仓库操作"></a>练习:远程仓库操作</h4><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">#########################1-将本地仓库推送到远程仓库</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">完成4.1、4.2、4.3、4.4的操作</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test01]添加远程仓库</span></span><br><span class="line">git remote add origin git@gitee.com/**/**.git</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test01]将master分支推送到远程仓库,并与远程仓库的master分支绑定关联关系</span></span><br><span class="line">git push --set-upstream origin master</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">##########################2-将远程仓库克隆到本地</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">将远程仓库克隆到本地git_test02目录下</span></span><br><span class="line">git clone git@gitee.com/**/**.git git_test02</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test02]以精简的方式显示提交记录</span></span><br><span class="line">git-log</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">##########################3-将本地修改推送到远程仓库</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test01]创建文件file03.txt</span></span><br><span class="line">略</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test01]将修改加入暂存区并提交到仓库,提交记录内容为：add file03</span></span><br><span class="line">git add .</span><br><span class="line">git commit -m 'add file03'</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test01]将master分支的修改推送到远程仓库</span></span><br><span class="line">git push origin master</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment">##########################4-将远程仓库的修改更新到本地</span></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">[git_test02]将远程仓库修改再拉取到本地</span></span><br><span class="line">git pull</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">以精简的方式显示提交记录</span></span><br><span class="line">git-log</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">查看文件变化(目录下也出现了file03.txt)</span></span><br><span class="line">略</span><br></pre></td></tr></table></figure><h4 id="移除远程仓库"><a href="#移除远程仓库" class="headerlink" title="移除远程仓库"></a>移除远程仓库</h4><ul><li>命令：<code>git remote rm <remote-name></code> = <code>git remote remove <remote-name></code></li></ul><h2 id="5、在Idea中使用Git"><a href="#5、在Idea中使用Git" class="headerlink" title="5、在Idea中使用Git"></a>5、在Idea中使用Git</h2><h3 id="5-1、在Idea中配置Git"><a href="#5-1、在Idea中配置Git" class="headerlink" title="5.1、在Idea中配置Git"></a>5.1、在Idea中配置Git</h3><p>安装好IntelliJ IDEA后，如果Git安装在默认路径下，那么idea会自动找到git的位置，如果更改了Git的安装位置则需要手动配置下Git的路径。选择File→Settings打开设置窗口，找到Version Control下的git选项：</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%9C%A8Idea%E4%B8%AD%E9%85%8D%E7%BD%AEGit.png" alt="在Idea中配置Git"></p><p>点击Test按钮,现在执行成功，配置完成</p><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E5%9C%A8Idea%E4%B8%AD%E6%88%90%E5%8A%9F%E9%85%8D%E7%BD%AEGit.png" alt="在Idea中成功配置Git"></p><h3 id="5-2、在Idea中操作Git"><a href="#5-2、在Idea中操作Git" class="headerlink" title="5.2、在Idea中操作Git"></a>5.2、在Idea中操作Git</h3><p>场景：本地已经有一个项目，但是并不是git项目，我们需要将这个放到码云的仓库里，和其他开发人员继续一起协作开发。</p><h4 id="5-2-1、创建项目远程仓库（参照4-3）"><a href="#5-2-1、创建项目远程仓库（参照4-3）" class="headerlink" title="5.2.1、创建项目远程仓库（参照4.3）"></a>5.2.1、创建项目远程仓库（参照4.3）</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E5%88%9B%E5%BB%BA%E9%A1%B9%E7%9B%AE%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93.png" alt="IDEA-创建项目远程仓库"></p><h4 id="5-2-2、初始化本地仓库"><a href="#5-2-2、初始化本地仓库" class="headerlink" title="5.2.2、初始化本地仓库"></a>5.2.2、初始化本地仓库</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E5%88%9D%E5%A7%8B%E5%8C%96%E6%9C%AC%E5%9C%B0%E4%BB%93%E5%BA%93.png" alt="IDEA-初始化本地仓库"></p><h4 id="5-2-3、设置远程仓库"><a href="#5-2-3、设置远程仓库" class="headerlink" title="5.2.3、设置远程仓库"></a>5.2.3、设置远程仓库</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E8%AE%BE%E7%BD%AE%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93.png" alt="IDEA-设置远程仓库"></p><h4 id="5-2-4、提交到本地仓库"><a href="#5-2-4、提交到本地仓库" class="headerlink" title="5.2.4、提交到本地仓库"></a>5.2.4、提交到本地仓库</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E6%8F%90%E4%BA%A4%E5%88%B0%E6%9C%AC%E5%9C%B0%E4%BB%93%E5%BA%93.png" alt="IDEA-提交到本地仓库"></p><h4 id="5-2-6、推送到远程仓库"><a href="#5-2-6、推送到远程仓库" class="headerlink" title="5.2.6、推送到远程仓库"></a>5.2.6、推送到远程仓库</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E6%8E%A8%E9%80%81%E5%88%B0%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93.png" alt="IDEA-推送到远程仓库"></p><h4 id="5-2-7、克隆远程仓库到本地"><a href="#5-2-7、克隆远程仓库到本地" class="headerlink" title="5.2.7、克隆远程仓库到本地"></a>5.2.7、克隆远程仓库到本地</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E5%85%8B%E9%9A%86%E8%BF%9C%E7%A8%8B%E4%BB%93%E5%BA%93%E5%88%B0%E6%9C%AC%E5%9C%B0.png" alt="IDEA-克隆远程仓库到本地"></p><h4 id="5-2-8、创建分支"><a href="#5-2-8、创建分支" class="headerlink" title="5.2.8、创建分支"></a>5.2.8、创建分支</h4><ul><li>最常规的方式<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E5%88%9B%E5%BB%BA%E5%88%86%E6%94%AF%EF%BC%88%E5%B8%B8%E8%A7%84%EF%BC%89.png" alt="IDEA-创建分支（常规）"></li><li>最强大的的方式<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E5%88%9B%E5%BB%BA%E5%88%86%E6%94%AF%EF%BC%88%E5%BC%BA%E5%A4%A7%EF%BC%89.png" alt="IDEA-创建分支（强大）"></li></ul><h4 id="5-2-9、切换分支及其他分支相关操作"><a href="#5-2-9、切换分支及其他分支相关操作" class="headerlink" title="5.2.9、切换分支及其他分支相关操作"></a>5.2.9、切换分支及其他分支相关操作</h4><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E5%88%87%E6%8D%A2%E5%88%86%E6%94%AF%E5%8F%8A%E5%85%B6%E4%BB%96%E5%88%86%E6%94%AF%E7%9B%B8%E5%85%B3%E6%93%8D%E4%BD%9C.png" alt="IDEA-切换分支及其他分支相关操作"></p><h4 id="5-2-11、解决冲突"><a href="#5-2-11、解决冲突" class="headerlink" title="5.2.11、解决冲突"></a>5.2.11、解决冲突</h4><ol><li>执行merge或pull操作时，可能发生冲突<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA-%E8%A7%A3%E5%86%B3%E5%86%B2%E7%AA%81.png" alt="IDEA-解决冲突"></li><li>冲突解决后加入暂存区<br>略</li><li>提交到本地仓库<br>略</li><li>推送到远程仓库<br>略</li></ol><h3 id="5-3、IDEA常用GIT操作入口"><a href="#5-3、IDEA常用GIT操作入口" class="headerlink" title="5.3、IDEA常用GIT操作入口"></a>5.3、IDEA常用GIT操作入口</h3><ol><li>第一张图上的快捷入口可以基本满足开发的需求。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA%E5%B8%B8%E7%94%A8GIT%E6%93%8D%E4%BD%9C%E5%85%A5%E5%8F%A31.png" alt="IDEA常用GIT操作入口"></li><li>第二张图是更多在IDEA操作git的入口。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA%E5%B8%B8%E7%94%A8GIT%E6%93%8D%E4%BD%9C%E5%85%A5%E5%8F%A32.png" alt="IDEA常用GIT操作入口"></li></ol><h3 id="5-4、场景分析"><a href="#5-4、场景分析" class="headerlink" title="5.4、场景分析"></a>5.4、场景分析</h3><p>基于我们后面的实战模式，我们做一个综合练习</p><p>当前的开发环境如下，我们每个人都对这个项目已经开发一段时间，接下来我们要切换成团队开发模<br>式。</p><p>也就是我们由一个团队来完成这个项目实战的内容。团队有组长和若干组员组成（组长就是开发中的项<br>目经理）。</p><p>所有操作都在idea中完成。</p><p>练习场景如下：</p><ol><li>由组长，基于本项目创建本地仓库；创建远程仓库，推送项目到远程仓库。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E7%BB%BC%E5%90%88%E7%BB%83%E4%B9%A0-1.png" alt="综合练习-1"></li><li>每一位组员从远程仓库克隆项目到idea中,这样每位同学在自己电脑上就有了一个工作副本，可以正<br>式的开始开发了。我们模拟两个组员(组员A、组员B)，克隆两个工作区。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E7%BB%BC%E5%90%88%E7%BB%83%E4%B9%A0-2.png" alt="综合练习-2"></li><li>组员A修改工作区,提交到本地仓库，再推送到远程仓库。组员B可以直接从远程仓库获取最新的代<br>码。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E7%BB%BC%E5%90%88%E7%BB%83%E4%B9%A0-3.png" alt="综合练习-3"></li><li>组员A和组员B修改了同一个文件的同一行，提交到本地没有问题，但是推送到远程仓库时，后一个<br>推送操作就会失败。<br>解决方法：需要先获取远程仓库的代码到本地仓库，编辑冲突，提交并推送代码。<br><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/%E7%BB%BC%E5%90%88%E7%BB%83%E4%B9%A0-4.png" alt="综合练习-4"></li></ol><h2 id="附-几条铁令"><a href="#附-几条铁令" class="headerlink" title="附:几条铁令"></a>附:几条铁令</h2><ol><li><strong>切换分支前先提交本地的修改</strong></li><li>代码及时提交，提交过了就不会丢</li><li>遇到任何问题都不要删除文件目录，第1时间找老师</li></ol><h2 id="附-疑难问题解决"><a href="#附-疑难问题解决" class="headerlink" title="附:疑难问题解决"></a>附:疑难问题解决</h2><h3 id="1-windows下看不到隐藏的文件（-bashrc、-gitignore）"><a href="#1-windows下看不到隐藏的文件（-bashrc、-gitignore）" class="headerlink" title="1. windows下看不到隐藏的文件（.bashrc、.gitignore）"></a>1. windows下看不到隐藏的文件（.bashrc、.gitignore）</h3><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/windows%E4%B8%8B%E6%9F%A5%E7%9C%8B%E9%9A%90%E8%97%8F%E6%96%87%E4%BB%B6.png" alt="windows下查看隐藏文件"></p><h3 id="2-windows下无法创建-ignore-bashrc文件"><a href="#2-windows下无法创建-ignore-bashrc文件" class="headerlink" title="2. windows下无法创建.ignore|.bashrc文件"></a>2. windows下无法创建.ignore|.bashrc文件</h3><p>这里以创建.ignore 文件为例：</p><ul><li>在git目录下打开gitbash</li><li>执行指令 <code>touch .gitignore</code></li></ul><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/windows下创建.ignore、.bashrc文件.png" alt="windows下创建.ignore、.bashrc文件"></p><h2 id="附：IDEA集成GitBash作为Terminal"><a href="#附：IDEA集成GitBash作为Terminal" class="headerlink" title="附：IDEA集成GitBash作为Terminal"></a>附：IDEA集成GitBash作为Terminal</h2><p><img src="https://gcore.jsdelivr.net/gh/Warma10032/image@main/blog/IDEA%E9%9B%86%E6%88%90GitBash%E4%BD%9C%E4%B8%BATerminal.png" alt="IDEA集成GitBash作为Terminal"></p>]]></content>
<categories>
<category> 学习笔记 </category>
</categories>
<tags>
<tag> 教程 </tag>
<tag> Git </tag>
<tag> Github </tag>
</tags>
</entry>
</search>