Jack Gallant's group at UC Berkley has done a lot of interesting fMRI decoding. The approach in their earlier papers (e.g., Kay et al, 2008 [0]) is neat because it uses some knowledge of how early visual areas represent information, rather than just flinging a decoder at it. They later extended this to movies (Nishimoto et al., 2011 [1]), and have subsequently moved a bit away from sensory information to "semantic" decoding. Jack has a TED talk about this [2] and Kendrick Kay has written some News and Views/Perspective pieces [e.g., 3] that might be interesting too, as it links to other groups' work.
I like their approach because it ties into neuroscience rather than just throwing ML at the wall and seeing what sticks, but there's also a huge cottage industry of "Blackbox ML" + MRI to do decoding. My sense is that this has moved slightly out of the limelight and into the stage where people incrementally improve it with newer ML techniques like GANs [4] and diffusion (this link), but it might just be me.
For sleep specifically, you might be interested in Horikawa et al. (2013)[5], who could predict words associated with dream contents (e.g., "Car" but didn't try to reconstruct them). There's also a very short perspective on it [6].
[0] Kay, K., Naselaris, T., Prenger, R. et al. Identifying natural images from human brain activity. Nature 452, 352–355 (2008). https://doi.org/10.1038/nature06713
[1] Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011). Reconstructing visual experiences from brain activity evoked by natural movies. Current biology : CB, 21(19), 1641–1646. https://doi.org/10.1016/j.cub.2011.08.031
[4] Seeliger, K., Güçlü, U., Ambrogioni, L., Güçlütürk, Y., & van Gerven, M. A. J. (2018). Generative adversarial networks for reconstructing natural images from brain activity. NeuroImage, 181, 775–785. https://doi.org/10.1016/j.neuroimage.2018.07.043
[5] Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013). Neural decoding of visual imagery during sleep. Science (New York, N.Y.), 340(6132), 639–642. https://doi.org/10.1126/science.1234330
Jack Gallant's group at UC Berkley has done a lot of interesting fMRI decoding. The approach in their earlier papers (e.g., Kay et al, 2008 [0]) is neat because it uses some knowledge of how early visual areas represent information, rather than just flinging a decoder at it. They later extended this to movies (Nishimoto et al., 2011 [1]), and have subsequently moved a bit away from sensory information to "semantic" decoding. Jack has a TED talk about this [2] and Kendrick Kay has written some News and Views/Perspective pieces [e.g., 3] that might be interesting too, as it links to other groups' work.
I like their approach because it ties into neuroscience rather than just throwing ML at the wall and seeing what sticks, but there's also a huge cottage industry of "Blackbox ML" + MRI to do decoding. My sense is that this has moved slightly out of the limelight and into the stage where people incrementally improve it with newer ML techniques like GANs [4] and diffusion (this link), but it might just be me.
For sleep specifically, you might be interested in Horikawa et al. (2013)[5], who could predict words associated with dream contents (e.g., "Car" but didn't try to reconstruct them). There's also a very short perspective on it [6].
[0] Kay, K., Naselaris, T., Prenger, R. et al. Identifying natural images from human brain activity. Nature 452, 352–355 (2008). https://doi.org/10.1038/nature06713
[1] Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011). Reconstructing visual experiences from brain activity evoked by natural movies. Current biology : CB, 21(19), 1641–1646. https://doi.org/10.1016/j.cub.2011.08.031
[2] https://www.youtube.com/watch?v=Ecvv-EvOj8M
[3] Kay, K. N., & Gallant, J. L. (2009). I can see what you see. Nature neuroscience, 12(3), 245. https://doi.org/10.1038/nn0309-245
[4] Seeliger, K., Güçlü, U., Ambrogioni, L., Güçlütürk, Y., & van Gerven, M. A. J. (2018). Generative adversarial networks for reconstructing natural images from brain activity. NeuroImage, 181, 775–785. https://doi.org/10.1016/j.neuroimage.2018.07.043
[5] Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013). Neural decoding of visual imagery during sleep. Science (New York, N.Y.), 340(6132), 639–642. https://doi.org/10.1126/science.1234330
[6] https://www.nature.com/articles/nmeth.2504