创建带有 LLMs 的演讲者备注
受Emilie在Helsing巴黎办公室的午餐学习计划的启发,我们昨天在慕尼黑开始了每周一次的午餐学习系列活动,主题是软件工程中的不变性概念。在未来的博客文章中,我会更多地谈及我们在指导和学习方面的方法(很可能也会公开发布午餐学习的演示材料),但是这篇文章我想重点讲述一下我为演讲设计主持人笔记的LLM实验。
创建高质量的演讲者备注的动机很简单:它们使他人能够复制给定的演示文稿,从而放大了在创建演示材料方面投入的时间和精力。不幸的是,演讲者备注通常是事后才考虑的,尤其是当演讲者对主题非常熟悉,不需要备注来进行演讲时。
在过去的几年里,我通常会请同事在我的演示中帮忙做笔记,这样我就可以事后将它们编辑成演讲者的备注。等一下。"做笔记"和"编辑"听起来像是LLM完全能够胜任的活动,对吧?让我们试一试!
步骤1:机器可读幻灯片
现成的LLM(如GPT-4)可以处理自然语言,也可以处理像Markdown这样的机器可读语言;这意味着它们应该能够编辑用Markdown编写的演示文稿。我非常喜欢用remark制作幻灯片,它具有简单明了的Markdown语法,足够支持常见内容(项目列表、代码格式化、图片、链接),支持多屏幕和演讲者备注,并且可以通过CSS进行自定义。例如,请查看我们的软件测试研讨会材料。
以下是有关不可变性讲座的前几张幻灯片的源代码
<!DOCTYPE html>
<html>
<head>
<title>Software building blocks – Immutability</title>
<meta charset=UTF-8 <textarea id="source">
---
# Immutability
- What is it?
- Why and when is it useful?
- Common patterns
- Limitations
---
class: center, middle
# What is it?
---
# Immutability defined
> *In object-oriented (OO) and functional programming, an immutable object (unchangeable object) is an object whose state cannot be modified after it is created.*
>
> *This is in contrast to a mutable object (changeable object), which can be modified after it is created.*
[[Source: Wikipedia]](https://en.wikipedia.org/wiki/Immutable_object)
---
# Examples
Python – Mutable or immutable?
```python
@dataclass
class Foo:
x: int
```
Rust – Mutable or immutable?
```rust
struct Foo {
x: u32
}
```
---
# Mutable and immutable Python dataclasses
.fullsize[![](img/python-frozen.png)]
---
# Mutable and immutable Rust instances
.fullsize[![](img/rust-mut.png)]
</textarea>
<script src="https://remarkjs.com/downloads/remark-latest.min.js">
</script>
<script>
var slideshow = remark.create();
</script>
</body>
</html>
备注库会将源代码呈现为如下幻灯片的形式:
![](/posts/1%24R9oYozZ9sACpQFHaSNrWfA.png)
第二步:进行演示
有了编写好的演示文稿,现在是给演示的时候了。请记得录制演示——我们需要在第三步使用音频。
步骤3: 创建一个记录
在2024年,语音转文本已经成为一种日用品。如果您使用MS Teams或类似平台录制了演讲,您很可能可以直接下载到转录文件。否则,您可以尝试使用基于OpenAI的Whisper模型的Buzz,或者像Gladia这样的基于网络的服务。正如我们将看到的,转录的质量并不太重要,因为在第4步骤中,我们将请LLM编辑并提炼演讲者注意事项。
以下是我本周演示的文字记录。请注意,它很随意和生硬... 包含了各种脱离语境的评论、部分保留的观众互动、未完成的句子、缺失或误解的单词,以及关于区块链中不可改变性的随机旁白(由观众的问题引发)。
6:29
Anyway, welcome to Session one of the Munich Lunch and Learn series.
6:35
All right.
6:36
Immutability.
6:40
What is immutability?
6:41
Any guesses?
6:43
I guess this is a bad format for audience participation because you're all eating.
6:47
Does anyone want to attempt a definition for what immutability is in the context of software engineering and coding?
6:57
Julian is collecting his thoughts.
6:59
Yeah, I'm collecting thoughts.
7:00
Go for it, I think.
7:01
Yeah, information that you create once and never such a change.
7:07
OK, could be in a database.
7:09
In the database.
7:10
OK, what?
7:12
What about Python?
7:13
Like what?
7:13
What about a thing?
7:15
What about a Python program could be immutable?
7:29
You said in creation.
7:31
Yep, you created once and then you would also not change the value.
7:35
You write a class, make an instance of that class.
7:38
And sometimes these classes are immutable in the sense that the values of that instance never change.
7:42
Cool.
7:45
The Wikipedia definition goes like this in object oriented and functional programming, so it apparently it doesn't apply to also types of programming, but an option object oriented functional programming.
7:55
An object is immutable if it's state cannot be modified after it's created.
7:59
So very close to what you said.
<snip; full transcript in the OpenAI chat at https://chat.openai.com/share/d32ae40b-df07-48b7-9ea5-2cadbf9c55d1>
步骤4:创建演示者备注
这是真的:我们可以向GPT-4提问吗?
- 解析文本
- 將以下英文文本翻譯成簡體中文時,請保留HTML結構,並將其轉化為合理的演示者備註。
- 将这些注释与幻灯片内容对齐
- 将注释插入幻灯片的Markdown,并使用正确的语法。
虽然我肯定不是一个敏捷工程师专家,但是我对我初次尝试的结果还是非常满意的:
I have a markdown file (written in remarkjs) representing the presenter slides
for a tech talk on "Immutability in Software Engineering". The file is split
into slides by "---" and contains presenter notes after the "???" separator.
For example, here are two slides with notes:
[beginning of markdown source code syntax example]
---
# Slide 1
- Some content
- More content
???
- A first presenter note
- And a second presenter note
---
# Slide 2
- More bullet points
And an image:
.fullsize[![](img/python-immut.png)]
???
- Example presenter note for the second slide
[end of markdown source code syntax example]
I will give you two inputs: the source code for the slides and the transcript
of the presentation I gave this morning. I would like you to write presenter
notes and insert them into the markdown source code as per the syntax example
above. The original slide content must be preserved exactly; you may only
insert additional content into the presenter notes after the ??? characters.
When editing the transcript, please remove any unintelligable content and
produce concise bullet-points with the most important aspects for each slide.
You can also augment the notes with additional aspects, caveats, or
explanations not present in the transcript. Please produce a single markdown
file containing the unedited slide content plus the presenter notes.
Here is the markdown for the slides:
[beginning of markdown source code]
---
# Immutability
- What is it?
- Why and when is it useful?
- Common patterns
- Limitations
SNIP: copy remaining slide source here
[end of markdown source code]
And here is the transcript:
[beging of transcript]
6:29
Anyway, welcome to Session one of the Munich Lunch and Learn series.
SNIP: copy rest of transcript here
[end of transcript]
ChatGPT的回应如下:
![](/posts/1%24b0JUIpTRhgAs23NM89zVdw.png)
完整的ChatGPT互动内容可以在OpenAI上找到。几个观察结果:
- 演讲者的笔记如要求一样简洁易读。
- 模型没有插入太多在文本中已存在的内容;我怀疑我们是否可以通过不同的提示来增加这一点。
- 幻灯片和主讲笔记的对齐完美,也就是说,该模型已经理解了文本中哪些部分与哪些幻灯片相关。
- 该模型生成了语法正确的Markdown,产生了一个干净的diff(见下文)。
- ChatGPT UI 不对三个反引号进行转义,导致界面混乱。请注意,ChatGPT 响应末尾的复制按钮会复制包含逐字逐句 Markdown 源代码的原始模型输出。
步骤5:合并差异
如果一切顺利,我们可以将模型生成的输出(使用ChatGPT响应末尾的复制按钮)复制粘贴到Markdown源代码中,并合并更改。
![](/posts/1%24irjQKVXZUHp8FNTvY9DyEw.png)
瞧,带有LLM动力的演讲备注幻灯片展示(按‘P’键在演示模式和演讲者模式之间切换):
![](/posts/1%24Il0C-ZpaMdZq82nMU9PM3g.png)
结论
这是一个有趣的实验,也是我将来一定会重复进行的工作流程,用于创建或编辑演示备注。随着LLM生态系统的发展,我也预计大众市场的演示工具,如Powerpoint或Keynote,将很快加入自动创建演示备注的功能。