After working for a few companies who (tried to) make ML products, I think one of the most important questions to ask is "who owns the data you are building models on?". It is way harder for a company to build good models off data they don't have complete access to and full knowledge about. The worst (and unfortunately common) scenario for companies trying to do AI is that data scientists don't have full access to all priors necessary to build good models, and the owners of the data don't really know much about it either. Spells death of company
Well said. It is a good tester of knowing how clear the people understand the problem themselves. Many people (particularly non-science background ones, no offence) don't know which kind of data they should access to build a product and, even worse, whether they have access to the data.
Not knowing the answer of this question = they do not know they are travelling on Titanic at best, if not a boat full of holes.
Yep, Question-Answering Semantic Role labeling is an interesting research project around crowdsourcing NLP datasets. https://dada.cs.washington.edu/qasrl/
This makes no sense. It's not just that you might, for example, die at 70 instead of 75 if you don't sleep. The point is that your quality of life during those 70 years would be horrible, for all the points listed in the article.
But you could use caloric restriction to get some of those years back. You're probably near rock-bottom grumpiness/misery anyway so the cost is minimal.
There's bound to be some level of trade-off. For example, you might be marginally better off health-wise to sleep for 8.5 rather than 8 hours but that's a lot of extra time wasted if the benefit is negligible.
Despite how much I dislike Facebook, this article started off decent, but took a strange (and disrepectful) turn into hacker culture and algorithms that was completely unfounded. Facebook addiction and the centralization of the Internet has literally nothing to do with the concept of algorithms. Sure, machine learning ventures at Facebook contribute to the need to track users and invade privacy, but that's not the fault of computer scientists working on these algorithms. That's the fault of Facebook's business model.
Its because its an excerpt from a book, I think. I totally agree, I'm on board with the point of the article but its written like an amateur Adam Curtis.
There's some cool things to note about Xi which make me more excited about it:
* It's being written by Raph Levien, known for his work on Inconsolata, Ghostscript, and font-rs[1] among other things.
* The design goals include incredibly high performance, beauty, reliability and developer friendliness[2].
* It's very fast at handling large files, though the editor is still missing a lot of functionality.
Raph was also interviewed on the New Rustacean podcast[3] and talks a fair bit about Xi.
I don’t see how this would be better than just Neovim with a few plugins. I switched from Vim to Neovim and love it. Highly configurable and rock solid for everything I have asked it to do. Although I miss things like a live Markdown editor with preview like what you see in VS Code, but that is a small price to pay for ultimate control over your editor.
I've just recently switched over to it for my day to day work. Admittedly there are rough edges, but I think it's fair to call it viable for sufficiently small values of viable. I expect to do a major push later in the year to polish it to the point of being really viable.
Yeah this is just click bait. I haven't looked at the model they deployed, but it's likely just not sophisticated enough to represent aspects like tone and frame of reference. It might just be a great word-based language model. That doesn't mean it's intentionally biasing against civility.
I am always skeptical of people trying to "emulate" the human brain in machine learning. We currently do not have the tools to accurately record and analyze the dynamics of networks of neurons in the brain, and any group that claims to advance ML with knowledge of the cortex is spouting bullshit. Modern advances in ML are driven by great engineering, not biological insight.
In the field of AI, our "great engineering" is not even a worthy comparison to what nature has achieved. Maybe there are a few more things to learn from it.
Also we've been studying the neocortex for a long time and have learned a lot more about it than most people realize.
While Demis Hassabis and probably some DeepMind researchers use some understandings from the neuroscience literature as an inspiration for their work, I am pretty sure that a majority of DeepMind researchers would rather use a combination of mathematics and trial and error experiments to build an intuition to guide the design of their next iteration of intelligent learning systems.
The point is that when mainstream ML begins to realize that there are limitations to the current cartoonish representations of neural networks, they then go back to the biology to see what they're missing.
However there are other companies, like Numenta, which realized decades ago that the current techniques will not be sufficient for general intelligence.
Numenta is not trying to emulate the brain like the Human Brain Project, they are aiming to learn the principles behind the neocortex and replicate it in software.
Again, I don't think most people know enough about the neocortex because if they did, we probably wouldn't be so quick to discard the only real example of intelligence we have.
>there are limitations to the current cartoonish representations of neural networks
How? on a broad level Deep Learning is the same as natural neural network. Signal in and then neuron decides to fire a signal out. The algorithms inside is what differentiates a human from a machine. As long as the algorithm can make intelligent decisions who cares how the human algorithm works. We are not trying to build a human brain, we are trying to build a better than human brain
To be more factual: a majority of papers published by researchers at DeepMind do not cite any result from the neuroscience literature in their bibliography. Instead they cite other papers from the Machine Learning community.
I recall a quote from some neuroscientist. That whenever he hears people say "we know nothing about the human brain", he wants to smack them with a 900 page neuroscience textbook.
I think the biggest issue is these domains are isolated and don't talk to each other. It's not that ML researchers couldn't be inspired by neuroscience research. They just don't know any.
I talked to a researcher outside of the mainstream who was obsessed with biologically plausible models. He got good results, but not SOTA.
However his main argument was that his methods were much faster and more data efficient than standard practice. E.g. they did online learning and didn't suffer from catastrophic forgetting. Didn't require supervision and labelled data.
Standard methods are optimized towards getting the most accuracy on benchmarks, and not necessarily under realistic conditions. Real brains don't get to save huge dataset and iterate over them later. They need to learn in real time and without forgetting previously learned knowledge. Given just a stream of unlabeled data. ANNs can't do this at all. Some biologically inspired models claim to be able to do this well.
>Real brains don't get to save huge dataset and iterate over them later. They need to learn in real time and without forgetting previously learned knowledge. Given just a stream of unlabeled data.
It's more than just that. Real brains need to optimize their internal data for action. The ultimate test of whether you've represented the world correctly is: but can you do stuff? Can you control an inverted pendulum (to pick a task) while constrained to have your representation updates be Lipschitz functions, with a Lipschitz constant based on your actuators' state (ie: speed, angle, force, etc)?
Note the trick here! Your representation doesn't have to be reconstructive itself (allowing for you to conditionally simulate only Lipschitz transformations), but the updates you perform on that transformation from sensory reafferant signals do need to change only at a bounded rate, because the physics of the thing you're moving actually have that property.
They allowed the connectome to control a lego robot [1] and it showed some signs of working around obstacles (robot backs up when it encounters a wall). Sounds simple, but now we can say that only 300 neurons are needed to be able to react to the environment in this way, which is a huge stepping stone in understanding.
Ok, but what functional knowledge did we gain? Not "run this neural net, and an interesting thing happens". How are those 300 neurons implementing interesting motor behavior? What behavior are they really implementing? What's the algorithm, what's its goal?
The connectome model only addresses the third and lowest levels of Marr's analysis of a cognitive/biological system.
1. Computational: What does the system aim to do? What problem does it solve?
2. Algorithmic: How does the system solve or approximately solve that problem? How does it accomplish its purpose as a part of the organism?
3. Implementation: How are cells and/or organs put together to implement that solution?
You can have a very accurate picture of (3), and still lack any solid knowledge about (2) or (1). You can also sometimes have a clear picture of (2) and (3), but overgeneralize and wind up with a near-tautological theory for (1) -- that's the accusation being thrown at certain parts of theoretical neuroscience today.
You talk like the project is over, there is still a lot to learn and we aren't going to figure out everything immediately. A simulation provides an amazing test bed for ideas though; now you can see it working, you can pause it, change things, do experiments, and so on.
