postcard/de/flavors.rs
1//! # Deserialization Flavors
2//!
3//! "Flavors" in `postcard` are used as modifiers to the serialization or deserialization
4//! process. Flavors typically modify one or both of the following:
5//!
6//! 1. The source medium of the deserialization, e.g. whether the data is serialized from a `[u8]` slice, or some other container
7//! 2. The format of the deserialization, such as if the original data is encoded in a COBS format, contains a CRC32 checksum
8//! appended to the message, etc.
9//!
10//! Flavors are implemented using the [`Flavor`] trait, which acts as a "middleware" for retrieving the bytes before they
11//! are passed to `serde` for deserialization
12//!
13//! Multiple flavors may be combined to obtain a desired combination of behavior and storage.
14//! When flavors are combined, it is expected that the storage flavor (such as [`Slice`]) is the innermost flavor.
15//!
16//! Custom flavors may be defined by users of the `postcard` crate, however some commonly useful flavors have been provided in
17//! this module. If you think your custom flavor would be useful to others, PRs adding flavors are very welcome!
18//!
19//! ## Usability
20//!
21//! Flavors may not always be convenient to use directly, as they may expose some implementation details of how the
22//! inner workings of the flavor behaves. It is typical to provide a convenience method for using a flavor, to prevent
23//! the user from having to specify generic parameters, setting correct initialization values, or handling the output of
24//! the flavor correctly. See `postcard::from_bytes()` for an example of this.
25//!
26//! ## When to use (multiple) flavors
27//!
28//! Combining flavors are nice for convenience, as they perform potentially multiple steps of
29//! serialization at one time.
30//!
31//! This can often be more memory efficient, as intermediate buffers are not typically required.
32//!
33//! ## When NOT to use (multiple) flavors
34//!
35//! The downside of passing deserialization through multiple steps is that it is typically slower than
36//! performing each step serially. Said simply, "cobs decoding while deserializing" is often slower
37//! than "cobs decode then deserialize", due to the ability to handle longer "runs" of data in each
38//! stage. The downside is that if these stages can not be performed in-place on the buffer, you
39//! will need additional buffers for each stage.
40//!
41//! Additionally, deserializating flavors can be more restrictive or difficult to work with than
42//! serialization flavors, as deserialization may require that the deserialized types borrow some
43//! portion of the original message.
44//!
45//! ## Examples
46//!
47//! ### Using a single flavor
48//!
49//! In the first example, we use the `Slice` flavor, to retrieve the serialized output from a `[u8]` slice.
50//! No other modification is made to the serialization process.
51//!
52//! ```rust
53//! use postcard::{
54//! de_flavors::Slice,
55//! Deserializer,
56//! };
57//! use serde::Deserialize;
58//!
59//! #[derive(Deserialize, Debug, PartialEq)]
60//! struct Tup(u8, u8, u8);
61//!
62//! let msg = [0x04, 0x00, 0x04, 0x01, 0x02, 0x03];
63//! let slice = Slice::new(&msg);
64//! let mut deserializer = Deserializer::from_flavor(slice);
65//! let t = Tup::deserialize(&mut deserializer).unwrap();
66//! assert_eq!(t, Tup(4, 0, 4));
67//! let remainder = deserializer.finalize().unwrap();
68//! assert_eq!(remainder, &[1, 2, 3]);
69//! ```
70
71use crate::{Error, Result};
72use core::marker::PhantomData;
73
74/// The deserialization Flavor trait
75///
76/// This is used as the primary way to decode serialized data from some kind of buffer,
77/// or modify that data in a middleware style pattern.
78///
79/// See the module level docs for an example of how flavors are used.
80pub trait Flavor<'de>: 'de {
81 /// The remaining data of this flavor after deserializing has completed.
82 ///
83 /// Typically, this includes the remaining buffer that was not used for
84 /// deserialization, and in cases of more complex flavors, any additional
85 /// information that was decoded or otherwise calculated during
86 /// the deserialization process.
87 type Remainder: 'de;
88
89 /// The source of data retrieved for deserialization.
90 ///
91 /// This is typically some sort of data buffer, or another Flavor, when
92 /// chained behavior is desired
93 type Source: 'de;
94
95 /// Obtain the next byte for deserialization
96 fn pop(&mut self) -> Result<u8>;
97
98 /// Returns the number of bytes remaining in the message, if known.
99 ///
100 /// # Implementation notes
101 ///
102 /// It is not enforced that this number is exactly correct.
103 /// A flavor may yield less or more bytes than the what is hinted at by
104 /// this function.
105 ///
106 /// `size_hint()` is primarily intended to be used for optimizations such as
107 /// reserving space for deserialized items, but must not be trusted to
108 /// e.g., omit bounds checks in unsafe code. An incorrect implementation of
109 /// `size_hint()` should not lead to memory safety violations.
110 ///
111 /// That said, the implementation should provide a correct estimation,
112 /// because otherwise it would be a violation of the trait’s protocol.
113 ///
114 /// The default implementation returns `None` which is correct for any flavor.
115 fn size_hint(&self) -> Option<usize> {
116 None
117 }
118
119 /// Attempt to take the next `ct` bytes from the serialized message
120 fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]>;
121
122 /// Complete the deserialization process.
123 ///
124 /// This is typically called separately, after the `serde` deserialization
125 /// has completed.
126 fn finalize(self) -> Result<Self::Remainder>;
127}
128
129/// A simple [`Flavor`] representing the deserialization from a borrowed slice
130pub struct Slice<'de> {
131 // This string starts with the input data and characters are truncated off
132 // the beginning as data is parsed.
133 pub(crate) cursor: *const u8,
134 pub(crate) end: *const u8,
135 pub(crate) _pl: PhantomData<&'de [u8]>,
136}
137
138impl<'de> Slice<'de> {
139 /// Create a new [Slice] from the given buffer
140 pub fn new(sli: &'de [u8]) -> Self {
141 Self {
142 cursor: sli.as_ptr(),
143 end: unsafe { sli.as_ptr().add(sli.len()) },
144 _pl: PhantomData,
145 }
146 }
147}
148
149impl<'de> Flavor<'de> for Slice<'de> {
150 type Remainder = &'de [u8];
151 type Source = &'de [u8];
152
153 #[inline]
154 fn pop(&mut self) -> Result<u8> {
155 if self.cursor == self.end {
156 Err(Error::DeserializeUnexpectedEnd)
157 } else {
158 unsafe {
159 let res = Ok(*self.cursor);
160 self.cursor = self.cursor.add(1);
161 res
162 }
163 }
164 }
165
166 #[inline]
167 fn size_hint(&self) -> Option<usize> {
168 Some((self.end as usize) - (self.cursor as usize))
169 }
170
171 #[inline]
172 fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> {
173 let remain = (self.end as usize) - (self.cursor as usize);
174 if remain < ct {
175 Err(Error::DeserializeUnexpectedEnd)
176 } else {
177 unsafe {
178 let sli = core::slice::from_raw_parts(self.cursor, ct);
179 self.cursor = self.cursor.add(ct);
180 Ok(sli)
181 }
182 }
183 }
184
185 /// Return the remaining (unused) bytes in the Deserializer
186 fn finalize(self) -> Result<&'de [u8]> {
187 let remain = (self.end as usize) - (self.cursor as usize);
188 unsafe { Ok(core::slice::from_raw_parts(self.cursor, remain)) }
189 }
190}
191
192/// Support for [`std::io`] or `embedded-io` traits
193#[cfg(any(
194 feature = "embedded-io-04",
195 feature = "embedded-io-06",
196 feature = "use-std"
197))]
198pub mod io {
199 use crate::{Error, Result};
200 use core::marker::PhantomData;
201
202 struct SlidingBuffer<'de> {
203 cursor: *mut u8,
204 end: *const u8,
205 _pl: PhantomData<&'de [u8]>,
206 }
207
208 impl<'de> SlidingBuffer<'de> {
209 pub fn new(sli: &'de mut [u8]) -> Self {
210 Self {
211 cursor: sli.as_mut_ptr(),
212 end: unsafe { sli.as_ptr().add(sli.len()) },
213 _pl: PhantomData,
214 }
215 }
216
217 #[inline]
218 fn size(&self) -> usize {
219 (self.end as usize) - (self.cursor as usize)
220 }
221
222 #[inline]
223 fn take_n(&mut self, ct: usize) -> Result<&'de mut [u8]> {
224 let remain = (self.end as usize) - (self.cursor as usize);
225 let buff = if remain < ct {
226 return Err(Error::DeserializeUnexpectedEnd);
227 } else {
228 unsafe {
229 let sli = core::slice::from_raw_parts_mut(self.cursor, ct);
230 self.cursor = self.cursor.add(ct);
231 sli
232 }
233 };
234
235 Ok(buff)
236 }
237
238 fn complete(self) -> Result<&'de mut [u8]> {
239 let remain = (self.end as usize) - (self.cursor as usize);
240 unsafe { Ok(core::slice::from_raw_parts_mut(self.cursor, remain)) }
241 }
242 }
243
244 /// Support for [`embedded_io`](crate::eio::embedded_io) traits
245 #[cfg(any(feature = "embedded-io-04", feature = "embedded-io-06"))]
246 pub mod eio {
247 use super::super::Flavor;
248 use super::SlidingBuffer;
249 use crate::{Error, Result};
250
251 /// Wrapper over a [`embedded_io`](crate::eio::embedded_io)::[`Read`](crate::eio::Read) and a sliding buffer to implement the [`Flavor`] trait
252 pub struct EIOReader<'de, T>
253 where
254 T: crate::eio::Read,
255 {
256 reader: T,
257 buff: SlidingBuffer<'de>,
258 }
259
260 impl<'de, T> EIOReader<'de, T>
261 where
262 T: crate::eio::Read,
263 {
264 /// Create a new [`EIOReader`] from a reader and a buffer.
265 ///
266 /// `buff` must have enough space to hold all data read during the deserialisation.
267 pub fn new(reader: T, buff: &'de mut [u8]) -> Self {
268 Self {
269 reader,
270 buff: SlidingBuffer::new(buff),
271 }
272 }
273 }
274
275 impl<'de, T> Flavor<'de> for EIOReader<'de, T>
276 where
277 T: crate::eio::Read + 'de,
278 {
279 type Remainder = (T, &'de mut [u8]);
280 type Source = &'de [u8];
281
282 #[inline]
283 fn pop(&mut self) -> Result<u8> {
284 let mut val = [0; 1];
285 self.reader
286 .read(&mut val)
287 .map_err(|_| Error::DeserializeUnexpectedEnd)?;
288 Ok(val[0])
289 }
290
291 #[inline]
292 fn size_hint(&self) -> Option<usize> {
293 Some(self.buff.size())
294 }
295
296 #[inline]
297 fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> {
298 let buff = self.buff.take_n(ct)?;
299 self.reader
300 .read_exact(buff)
301 .map_err(|_| Error::DeserializeUnexpectedEnd)?;
302 Ok(buff)
303 }
304
305 /// Return the remaining (unused) bytes in the Deserializer
306 fn finalize(self) -> Result<(T, &'de mut [u8])> {
307 let buf = self.buff.complete()?;
308 Ok((self.reader, buf))
309 }
310 }
311 }
312
313 /// Support for [`std::io`] traits
314 #[allow(clippy::module_inception)]
315 #[cfg(feature = "use-std")]
316 pub mod io {
317 use super::super::Flavor;
318 use super::SlidingBuffer;
319 use crate::{Error, Result};
320
321 /// Wrapper over a [`std::io::Read`] and a sliding buffer to implement the [Flavor] trait
322 pub struct IOReader<'de, T>
323 where
324 T: std::io::Read,
325 {
326 reader: T,
327 buff: SlidingBuffer<'de>,
328 }
329
330 impl<'de, T> IOReader<'de, T>
331 where
332 T: std::io::Read,
333 {
334 /// Create a new [`IOReader`] from a reader and a buffer.
335 ///
336 /// `buff` must have enough space to hold all data read during the deserialisation.
337 pub fn new(reader: T, buff: &'de mut [u8]) -> Self {
338 Self {
339 reader,
340 buff: SlidingBuffer::new(buff),
341 }
342 }
343 }
344
345 impl<'de, T> Flavor<'de> for IOReader<'de, T>
346 where
347 T: std::io::Read + 'de,
348 {
349 type Remainder = (T, &'de mut [u8]);
350 type Source = &'de [u8];
351
352 #[inline]
353 fn pop(&mut self) -> Result<u8> {
354 let mut val = [0; 1];
355 self.reader
356 .read(&mut val)
357 .map_err(|_| Error::DeserializeUnexpectedEnd)?;
358 Ok(val[0])
359 }
360
361 #[inline]
362 fn size_hint(&self) -> Option<usize> {
363 Some(self.buff.size())
364 }
365
366 #[inline]
367 fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> {
368 let buff = self.buff.take_n(ct)?;
369 self.reader
370 .read_exact(buff)
371 .map_err(|_| Error::DeserializeUnexpectedEnd)?;
372 Ok(buff)
373 }
374
375 /// Return the remaining (unused) bytes in the Deserializer
376 fn finalize(self) -> Result<(T, &'de mut [u8])> {
377 let buf = self.buff.complete()?;
378 Ok((self.reader, buf))
379 }
380 }
381 }
382}
383
384////////////////////////////////////////
385// CRC
386////////////////////////////////////////
387
388/// This Cyclic Redundancy Check flavor applies [the CRC crate's `Algorithm`](https://docs.rs/crc/latest/crc/struct.Algorithm.html) struct on
389/// the serialized data.
390///
391/// The flavor will check the CRC assuming that it has been appended to the bytes.
392/// CRCs are used for error detection when reading data back.
393/// Requires the `crc` feature.
394///
395/// More on CRCs: <https://en.wikipedia.org/wiki/Cyclic_redundancy_check>.
396#[cfg(feature = "use-crc")]
397#[cfg_attr(docsrs, doc(cfg(feature = "use-crc")))]
398pub mod crc {
399 use core::convert::TryInto;
400
401 use crc::Digest;
402 use crc::Width;
403 use serde::Deserialize;
404
405 use super::Flavor;
406 use super::Slice;
407
408 use crate::Deserializer;
409 use crate::Error;
410 use crate::Result;
411 use paste::paste;
412
413 /// Manages CRC modifications as a flavor.
414 pub struct CrcModifier<'de, B, W>
415 where
416 B: Flavor<'de>,
417 W: Width,
418 {
419 flav: B,
420 digest: Digest<'de, W>,
421 }
422
423 impl<'de, B, W> CrcModifier<'de, B, W>
424 where
425 B: Flavor<'de>,
426 W: Width,
427 {
428 /// Create a new Crc modifier Flavor.
429 pub fn new(bee: B, digest: Digest<'de, W>) -> Self {
430 Self { flav: bee, digest }
431 }
432 }
433
434 macro_rules! impl_flavor {
435 ($( $int:ty ),*) => {
436 $(
437 paste! {
438 impl<'de, B> Flavor<'de> for CrcModifier<'de, B, $int>
439 where
440 B: Flavor<'de>,
441 {
442 type Remainder = B::Remainder;
443
444 type Source = B::Source;
445
446 #[inline]
447 fn pop(&mut self) -> Result<u8> {
448 match self.flav.pop() {
449 Ok(byte) => {
450 self.digest.update(&[byte]);
451 Ok(byte)
452 }
453 e @ Err(_) => e,
454 }
455 }
456
457 #[inline]
458 fn size_hint(&self) -> Option<usize> {
459 self.flav.size_hint()
460 }
461
462 #[inline]
463 fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> {
464 match self.flav.try_take_n(ct) {
465 Ok(bytes) => {
466 self.digest.update(bytes);
467 Ok(bytes)
468 }
469 e @ Err(_) => e,
470 }
471 }
472
473 fn finalize(mut self) -> Result<Self::Remainder> {
474 match self.flav.try_take_n(core::mem::size_of::<$int>()) {
475 Ok(prev_crc_bytes) => match self.flav.finalize() {
476 Ok(remainder) => {
477 let crc = self.digest.finalize();
478 let le_bytes = prev_crc_bytes
479 .try_into()
480 .map_err(|_| Error::DeserializeBadEncoding)?;
481 let prev_crc = <$int>::from_le_bytes(le_bytes);
482 if crc == prev_crc {
483 Ok(remainder)
484 } else {
485 Err(Error::DeserializeBadCrc)
486 }
487 }
488 e @ Err(_) => e,
489 },
490 Err(e) => Err(e),
491 }
492 }
493 }
494
495 /// Deserialize a message of type `T` from a byte slice with a Crc. The unused portion (if any)
496 /// of the byte slice is not returned.
497 pub fn [<from_bytes_ $int>]<'a, T>(s: &'a [u8], digest: Digest<'a, $int>) -> Result<T>
498 where
499 T: Deserialize<'a>,
500 {
501 let flav = CrcModifier::new(Slice::new(s), digest);
502 let mut deserializer = Deserializer::from_flavor(flav);
503 let r = T::deserialize(&mut deserializer)?;
504 let _ = deserializer.finalize()?;
505 Ok(r)
506 }
507
508 /// Deserialize a message of type `T` from a byte slice with a Crc. The unused portion (if any)
509 /// of the byte slice is returned for further usage
510 pub fn [<take_from_bytes_ $int>]<'a, T>(s: &'a [u8], digest: Digest<'a, $int>) -> Result<(T, &'a [u8])>
511 where
512 T: Deserialize<'a>,
513 {
514 let flav = CrcModifier::new(Slice::new(s), digest);
515 let mut deserializer = Deserializer::from_flavor(flav);
516 let t = T::deserialize(&mut deserializer)?;
517 Ok((t, deserializer.finalize()?))
518 }
519 }
520 )*
521 };
522 }
523
524 impl_flavor![u8, u16, u32, u64, u128];
525}