Mathematics is characterized by defining and studying various “mathematical objects”, such as sets, numbers, functions, graphs, sequences, polynomials, equations, Turing machines and complexity classes.
But two questions remain:
- How can a mathematician be sure that the object of conversation is in fact a mathematical object, and as such that the mathematician is justified in using the terminology and methods of dealing with mathematical objects?
- How does one ensure that one's definition of a mathematical object is unambiguous?
Even though we only study sets, the surprising thing is that we can still prove things about properties of numbers, functions, graphs, and all the other “mathematical objects” we wanted to study. Sets are so versatile that we can always make some construction out of them with properties that mirror those of a given mathematical object.
Thus, we don’t require a new ontological entity for each mathematical object, because we can simply redefine all our terms about the object and its properties so that they refer to certain sets and their properties.
Since sets are the only mathematical objects, you might still ask the one ontological question about mathematical objects left to pose in modern math: What is a set? But then, what is “what is”? Generally one answers an ontological question with the name of an entity, so any pure mathematical answer to this question must be circular.
So we leave the question of “what is a set” unanswered. Until further notice, you don’t know what sets are, you just define all your mathematical objects in terms of them.
This is all the philosophical underpinning you will need to start thinking about naïve set theory. We should just keep in mind what the source of our naiveté is: since we aren’t defining what a set is, we don’t say how to decide which definitions of sets are valid, and which are not.
But the hole we left in naïve set theory, that we didn’t say how to decide which definitions of sets are valid, allows us to construct a profound paradox (Bertrand Russel’s): Let S be the set of all sets which are not members of themselves. Is S a set? Well, we said “let S be a set”, so that should be enough – you don’t have any grounds to argue that it isn’t. Is S a member of itself? By the law of the excluded middle, you must believe that the answer is either yes or no. By the definition of S, you must also believe the opposite conclusion. But by the law of non-contradiction, you can’t do that. So the constraints of rational thinking make nonsense out of naïve set theory.
When we study axiomatic set theory, we define properties that a set must satisfy, instead of just letting intuition decide which definitions are valid sets and which aren’t. Axiomatizing set theory introduces a stunning array of counterintuitive results, but still enables us to avoid all known paradoxes.
What answer do we give to the question of “what is a set” in axiomatic set theory? Assuming that the axioms of set theory don’t contradict each other, Kurt Gödel’s completeness theorem tells us that axiomatic set theory has a model – meaning, in another meta-theory of sets which is a foundation for the study of axiom systems, there exists a meta-set whose elements satisfy our axioms' conditions of being sets. So we can say that those elements are the sets. But what kind of mathematical object is a model? It’s a set: not a set in our axiomatic set theory, but a set in the set meta-theory that underpins model theory which underpins the original axiomatic set theory.
Then what about meta-sets in the meta-theory of sets? All we can do is construct a model for one axiomatic set theory within another, and add arbitrarily many levels to the hierarchy of meta-sets inside meta-meta-sets.
If we look at any given level of meta-set in this hierarchy and ask what its definition is, there are two possibilities: either the set is an element of a model of an axiomatic set theory, or the question has no mathematical answer, because the set is the ontological foundation of the highest level model theory.
When we answered the question of what sets are in axiomatic set theory, we forced ourselves into a dead end by saying that they were elements of a model of set theory. But we can also give a second answer: sets are the symbols we write on paper as we mechanically apply inference rules to the set axioms (which are also symbols). So this is what we mathematicians do: define everything in terms of sets, define an axiomatic set theory which avoids all known paradoxes, and then field philosophical challenges by pretending to be blind mechanical theorem derivers. Then when the challenger goes away, we go back to abstracting and deriving meta-theorems.
If you want to talk about sets, and you don’t want to be stuck without a definition for sets at the highest level of the model hierarchy, then you have no choice but to take the notion of a set out of the scope of ontological study. You have to believe that every discussion about sets is shorthand for a discussion of symbols which purport to describe the sets. But at the highest level of axiomatic set theory, the one the mathematician writes in, the symbols can’t really be talking about anything.
It isn’t all that surprising that we initially reached an ontological dead end when we asked the question of what is a set. After all, every definition is made up of words, and there are only finitely many words. Thus, any chain of “what is” questions must end, or be answered with a circular definition.
And of course, we are still working our way down such a chain of questions. For what exactly are the “inference rules” and “strings of symbols” with which we confidently work? This is another discussion. But compared to the original question of what a set is, this is a discussion which seems quite alright for a mathematician to leave to a philosopher. A discussion of sets seems to strike much closer to the foundation of mathematics than a discussion of the mechanical execution of rules. Thus, we should be content to proceed with mathematics as usual, while leaving the philosophers to address a topic in the non-mathematical realm of ontological inquiry.