Reading Note

Chapter 1 GPT是自回归模型：像GPT这样的解码器式模型，它们生成文本的方式是一次预测一个词 (one word at a time)。也就是说，它会根据已经生成的词语序列来预测下一个最可能出现的词语，然后将这个预测到的词语加入到序列中，再进行下一步的预测，如此循环。这种依赖于自身先前输出进行下一步预测的特性，使得这类模型被称为“自回归模型”（Autoregressive model）。 Chapter 2 Word2Vec trained neural network architecture to generate word embeddings by predicting the context of a word given the target word or vice versa. The main idea behind Word2Vec is that words that appear in similar contexts tend to have similar meanings. LLMs commonly produce their own embeddings that are part of the input layer and are updated during training. The advantage of optimizing the embeddings as part of the LLM training instead of using Word2Vec is that the embeddings are optimized to the specific task and data at hand. ...

Reading Note1 General Advice on Technical Writing One of the most crucial aspects of writing a good technical paper is what I call maintaining user state. Like a good operating system, the writer should ensure that the (mental) state of the user (i.e. reader) is kept coherent. Important It means that the paper systematically builds up the reader’s understanding and knowledge of the work, starting from a reasonable initial state. ...

55: Mixing up concurrency and parallelism Concurrency enables parallelism. Concurrency provides a structure to solve a problem with parts that may be parallelized. Quote “Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.” ...

48: Panicking Panicking in Go should be used sparingly. We have seen two prominent cases, one to signal a programmer error and another where our application fails to create a mandatory dependency. 49: Ignoring when to wrap an error In general, the two main use cases for error wrapping are the following: Adding additional context to an error Marking an error as a specific error if err != nil { return fmt.Errorf("bar failed: %w", err) } This code wraps the source error to add additional context without having to create another error type. ...

42: Not knowing which type of receiver to use In Go, we can attach either a value or a pointer receiver to a method. With a value receiver, Go makes a copy of the value and passes it to the method. Any changes to the object remain local to the method. The original object remains unchanged. On the other hand, with a pointer receiver, Go passes the address of an object to the method. Intrinsically, it remains a copy, but we only copy a pointer, not the object itself (passing by reference doesn’t exist in Go). Any modifications to the receiver are done on the original object. ...

36: Not understanding the concept of a rune The len built-in function applied on a string doesn’t return the number of characters; it returns the number of bytes. What if we want to get the number of runes in a string, not the number of bytes? How we can do this depends on the encoding. In the previous example, because we assigned a string literal to s, we can use the unicode/utf8 package: ...

2 Unnecessary nested code 能够提前返回的特殊判定就提前返回 if xxx { // .... return err }else{ // do X } 总是能够写为 if xxx{ return err } // do X Make expected execution flow clear 5 Interface pollution Quote The bigger the interface, the weaker the abstraction. ...

17 Creating confusion with octal literals In Go, an integer literal starting with 0 is considered an octal integer (base 8). Note the integer literal representations in Golang: Binary - Uses a 0b or 0B prefix (for example, 0b100 is equal to 4 in base 10) Octal - Uses a 0 or 0o prefix (for example, 0o10 is equal 8 in base 10) Hexadecimal - Uses an 0x or 0X prefix (for example, 0xF is equal to 15 in base 10) Imaginary - Uses an i suffix (for example, 3i) 20 Not understanding slice length and capacity In Go, a slice is backed by an array. That means the slice’s data is stored contiguously in an array data structure. A slice also handles the logic of adding an element if the backing array is full or shrinking the backing array if it’s almost empty. ...

30: Ignoring the fact that elements are copied in range loops type account struct { balance float32 } accounts := []account{ {balance: 100.}, {balance: 200.}, {balance: 300.}, } for _, a := range accounts { a.balance += 1000 } // Output: [{100} {200} {300}] In this example, the range loop does not affect the slice’s content. In Go, everything we assign is a copy: ...

Chapter 3 Types Arrays 不支持动态开数组： // an array of 100 int32 elements, all set to 1 let mut arr1 = [1;100]; // correct let n = 100; let mur arr2 = [1;c]; // error Vectors let mut arr = vec![0,6,4,8,1]; arr.sort() // increasing order arr.sort_by(|a,b| b.cmp(a)) // decreasing order If you know the number of elements a vector will need in advance, instead of Vec::new you can call Vec::with_capacity to create a vector with a buffer large enough to hold them all, right from the start. ...

本章是这本书最酣畅淋漓的一章，涉及到了一致性和共识问题的方方面面，知识点多而不失条理。第一部分先讲了 Linearizability, 为后面的知识点做铺垫。到了 “Ordering Guarantees” 这一小节，从因果关系的带来的 “Happened Before” 的关系开始讲起，讲到了序列号和 Lamport Timestamp，提出来 Lamport Timestamp 的一个缺点：无法在某事件发生时判断是否有冲突，然后引出了全序关系广播，在全序关系广播中又讲到了和 Linearizable 之间的等价关系，最后引出共识算法。太精彩了，值得反复阅读！ Consistency Guarantees Most replicated databases provide at least eventual consistency, which means that if you stop writing to the database and wait for some unspecified length of time, then eventually all read requests will return the same value. A better name for eventual consistency may be convergence, as we expect all replicas to eventually converge to the same value. ...

Faults and Partial Failures An individual computer with good software is usually either fully functional or entirely broken, but not something in between. Thus, computers hide the fuzzy physical reality on which they are implemented and present an idealized system model that operates with mathematical perfection. In a distributed system, there may well be some parts of the system that are broken in some unpredictable way, even though other parts of the system are working fine. This is known as a partial failure. The difficulty is that partial failures are nondeterministic: if you try to do anything involving multiple nodes and the network, it may sometimes work and sometimes unpredictably fail. ...