Inductive reasoning is, to put it simply, reasoning from a specific case to a more general case. Most life depends on water to exist, so it seems that any new life form we discover will likely depend on it too. Induction differs from deduction in providing less robust conclusions. Deductive logic deals with possibility, while inductive logic deals with probability. We use induction to make inferences about the world around us all the time. It can be found in marketing and advertising, history, psychology, sports, and even science.
The problem of induction is commonly associated with David Hume, who questioned how induction might be justified in his 1748 work, An Enquiry Concerning Human Understanding. What is it that permits us to generalize from a particular, to connect our experience of life depending on water with the proposition or expectation that new life forms will depend on water as well? Hume noted that we make this connection not by deduction, but by induction, the very method we're employing
Centuries prior, Sextus Empiricus gave a brief and fitting summary of the problem:
When they propose to establish the universal from the particulars by means of induction, they will effect this by a review of either all or some of the particulars. But if they review some, the induction will be insecure, since some of the particulars omitted in the induction may contravene the universal; while if they are to review all, they will be toiling at the impossible, since the particulars are infinite and indefinite.
The problem of induction relates to what is usually called "the uniformity of nature". Science and logic seem to rely on the assumption that the natural world is in such a state of uniformity as to allow us to make reasonable inferences. If this assumption is untrue, then a great deal of what we think we know is wrong. But if this assumption is true, it would seem to call for an explanation, one that induction itself can't provide without begging the question.
Several responses have been made since Hume, with the most notable being Karl Popper. As a philosopher of science, Popper's approach primarily centers around scientific method, which he saw as bearing the responsibility of finding and correcting errors, rather than making inductive inferences. In fact, he argued that induction is a mere myth, and the way by which we actually create knowledge is through conjecture and criticism. Popper's solution has received plenty of criticism too, with the most substantial being that it doesn't really solve the problem of induction. His theory of falsification can go to show that, based on past experience, some scientific ideas are false, but it can't demonstrate when a scientific idea is true, a point he even conceded (Popper, 1979).
There is also the response of Hans Reichenbach, who offered a pragmatic justification of induction. His argument is strikingly familiar to Pascal's Wager in that he suggests that if we go with induction and it turns out to be true, we stand a better chance of success then if we were to go with any alternative reasoning, regardless of whether induction is true or not. Like Pascal's Wager, though, this pragmatic approach is only valuable in that it might point us to a useful strategy; it cannot tell us whether or not that strategy is founded on anything true (unless one accepts a pragmatist theory of truth, which comes with its own additional objections).
I am one of many people who concedes that the problem of induction has yet to be satisfied on epistemic grounds. Things like the uniformity of nature are really assumptions we make based on inferences for which we do not have solid justification. This is a real problem in philosophy.
Does the theist manage to avoid this problem by positing the existence of a god? In an article critiquing secular responses to the problem of induction, author James N. Anderson states at the tail end, "Of course, a Person for whom universal a priori knowledge of the very constitution of the universe is attainable (and perhaps even essential) would be an invaluable ally in such an epistemological predicament — especially so if that Person were inclined toward revelation of Himself and His universe." (Anderson, 2000)
Numerous assumptions are packed into this single sentence, but it is quite typical of the standard theistic belief regarding justification of induction, I've found. God created the universe and knows all about it, god has revealed himself and his works to human beings, and human beings like Anderson have been able to comprehend it as his god intended. These beliefs are fervently defended by apologists as being more than simple assumptions, but the ways in which they are defended are telling.
How do you know god created this universe? Look at the complexity, beauty, and order of nature, they might say. How do you know god has revealed himself to us in scripture? Look at the inerrancy of a text like the bible and the fulfilled prophecies in it. How do you know you've accurately received the revelation of god? I feel his presence and see confirmations of my faith in my life.
These are not ad hoc defenses of theistic beliefs. They make frequent appearances in the apologetic efforts of men like William Lane Craig, Gary Habermas, Josh McDowell, Alister McGrath, N.T. Wright, and many, many others. The beliefs that Anderson and his fellow theists see as giving justification to their use of induction are themselves justified by induction. There is no avoidance of the problem by tossing a deity into the mix and pretending that its existence would resolve everything. Note that even if the theist's god does exist, he is still left with the responsibility of justifying his own beliefs, a task which he undertakes by inductive reasoning.
In short, there is at present no escape from the problem of induction, not for theists or non-theists. When I engage in debates over this matter with believers, I call attention to this fact and invite them to see something like the problem of induction as a philosophical question to be tackled in mutual effort. We're both affected by this, so let's stop pointing fingers and look for a solution instead of trying to gain a foothold over the other. If induction has a secular solution, it will be available to theists as well, and should make no significant dent in their faith. But if the only thing you're interested in is a 'solution' that permits you to taunt the other side of the fence about how superior your worldview is, then it's no wonder your arguments revolve so much around the consistency of views than around the truth of those views. The irony is that consistency is a criterion that very often (perhaps always?) employs induction.
So what do we make of this problem? Is it a cause to stop drawing inferences altogether, to worry that life has no meaning, and to go sit in a dark corner and contemplate suicide? No, I dare say it's not. However, it is a cause to make us think more carefully about the inferences we draw, to consider that life may not have some of the meanings we ascribe to it, and to open ourselves up to new and different possibilities. Maybe induction is not perfectly justifiable because we tend to think of it in the wrong way, as if it ought to be more deductive or more absolute. Probability theory is not about making flawless predictions, but it can and does yield fairly reliable and practical results time after time, in discipline after discipline. Thus, I see our lack of a satisfactory answer to the problem of induction as no cause for drastic alarm. It is nonetheless a problem that merits interest, so that we don't forget our limitations and we continue to use caution in the conclusions we come to.
Anderson, Secular Responses to the Problem of Induction, Proginosko.com (2000).
Popper, Objective Knowledge: An Evolutionary Approach (Clarendon Press, 1979), p. 7