- AMS :: Proceedings of the American Mathematical Society
- Shop by category
- An Introduction to Mathematics of Emerging Biomedical Imaging
- Shop with confidence
Mikhailov, Equations aux Derivees Partielles , French transl. Wu, J. Yin and C. Zhizhiashshvilli , Some problems in the theory of simple and multiple trigonometric series, Russian Mathematical Surveys , 28 , Zuazua, Controllability and observability of partial differential equations: Some results and open problems, in Handbook of Differential Equations: Evolutionary Equations , Vol. Download as PowerPoint slide.
AMS :: Proceedings of the American Mathematical Society
Michael V. Nguyen , Hui Liu. A globally convergent numerical method for a 3D coefficient inverse problem with a single measurement of multi-frequency data. Determining the waveguide conductivity in a hyperbolic equation from a single measurement on the lateral boundary. Shitao Liu. Recovery of the sound speed and initial displacement for the wave equation by means of a single Dirichlet boundary measurement.
Eva Sincich , Mourad Sini. Local stability for soft obstacles by a single measurement. Mohsen Tadi. A computational method for an inverse problem in a parabolic system. On an inverse problem for fractional evolution equation. Sebastian Acosta. Recovery of the absorption coefficient in radiative transport from a single measurement. Gen Nakamura , Michiyuki Watanabe. An inverse boundary value problem for a nonlinear wave equation. Mourad Choulli , Aymen Jbalia. The problem of detecting corrosion by an electric measurement revisited. Patrick Martinez , Judith Vancostenoble.
The cost of boundary controllability for a parabolic equation with inverse square potential. A fast direct imaging method for the inverse obstacle scattering problem with nonlinear point scatterers.
Renormalizations of circle hoemomorphisms with a single break point. A phaseless inverse scattering problem for the 3-D Helmholtz equation. Kenichi Sakamoto , Masahiro Yamamoto. Inverse source problem with a final overdetermination for a fractional diffusion equation. John C. Schotland , Vadim A. Fourier-Laplace structure of the inverse scattering problem for the radiative transport equation.
A Carleman estimate for the linear shallow shell equation and an inverse source problem. Jaan Janno , Kairi Kasemets. Uniqueness for an inverse problem for a semilinear time-fractional diffusion equation. Li Liang. Global Carleman estimate on a network for the wave equation and application to an inverse problem. An inverse obstacle problem for the wave equation in a finite time domain. American Institute of Mathematical Sciences. Previous Article On state-dependent sweeping process in Banach spaces. Well-posedness and longtime behavior for a singular phase field system with perturbed phase dynamics.
Keywords: Inverse problem for parabolic equation , single point measurement. The recovery of a parabolic equation from measurements at a single point. References:  Sh. Google Scholar  H. Google Scholar  S. It is true that some mathematical scientists primarily prove theorems, while others primarily create and solve models, and professional reward systems need to take that into account. But any given individual might move between these modes of research, and many areas of specialization can and do include both kinds of work.
Overall, the array of mathematical sciences share a commonality of experience and thought processes, and there is a long history of insights from one area becoming useful in another. Thus, the committee concurs with the following statement made in the International Review of Mathematical Sciences Section 3. A long-standing practice has been to divide the mathematical sciences into categories that are, by implication, close to disjoint.
Furthermore, such distinctions can create unnecessary barriers and tensions within the mathematical sciences community by absorbing energy that might be expended more productively. In fact, there are increasing overlaps and beneficial interactions between different areas of the mathematical sciences. What is this commonality of experience that is shared across the mathematical sciences?
The mathematical sciences aim to understand the world by performing formal symbolic reasoning and computation on abstract structures. One aspect of the mathematical sciences involves unearthing and understanding deep relationships among these abstract structures. Another aspect involves capturing certain features of the world by abstract structures through the process of modeling, performing formal reasoning on these abstract structures or using them as a framework for computation, and then reconnecting back to make predictions about the world—often, this is an iterative process.
A related aspect is to use abstract reasoning and structures to make inferences about the world from data. This is linked to the quest to find ways to turn empirical observations into a means to classify, order, and understand reality—the basic promise of science. Through the mathematical sciences, researchers can construct a body of knowledge whose interrelations are understood and where whatever understanding one needs can be found and used.
The mathematical sciences also serve as a natural conduit through which concepts, tools, and best practices can migrate from field to field. A further aspect of the mathematical sciences is to investigate how to make the process of reasoning and computation as efficient as possible and to also characterize their limits. It is crucial to understand that these different aspects of the mathematical sciences do not proceed in isolation from one another. On the contrary, each aspect of the effort enriches the others with new problems, new tools, new insights, and—ultimately—new paradigms.
Put this way, there is no obvious reason that this approach to knowledge should have allowed us to understand the physical world. Yet the entire. The traditional areas of the mathematical sciences are certainly included. But many other areas of science and engineering are deeply concerned with building and evaluating mathematical models, exploring them computationally, and analyzing enormous amounts of observed and computed data. These activities are all inherently mathematical in nature, and there is no clear line to separate research efforts into those that are part of the mathematical sciences and those that are part of computer science or the discipline for which the modeling and analysis are performed.
The number of interfaces has increased since the time of Figure , and the mathematical sciences themselves have broadened in response. The academic science and engineering enterprise is suggested by the right half of the figure, while broader areas of human endeavor are indicated on the left. Within the academy, the mathematical sciences are playing a more integrative and foundational role, while within society more broadly their impacts affect all of us—although that is often unappreciated because it is behind the scenes. It does not attempt to represent the many other linkages that exist between academic disciplines and between those disciplines and the broad endeavors on the left, only because the full interplay is too complex for a two-dimensional schematic.
It is the collection of people who are advancing the mathematical sciences discipline. Some members of this community may be aligned professionally with two or more disciplines, one of which is the mathematical sciences. This alignment is reflected, for example, in which conferences they attend, which journals they publish in, which academic degrees they hold, and which academic departments they belong to. The collection of people in the areas of overlap is large. It includes statisticians who work in the geosciences, social sciences, bioinformatics, and other areas that, for historical reasons, became specialized offshoots of statistics.
It includes some fraction of researchers in scientific computing and computational science and engineering. It includes number theorists who contribute to cryptography, and real analysts and statisticians who contribute to machine learning. It includes operations researchers, some computer scientists, and physicists, chemists, ecologists, biologists, and economists who rely on sophisticated mathematical science approaches. Some of the engineers who advance mathematical models and computational simulation are also included. It is clear that the mathematical sciences now extend far beyond the definitions implied by the institutions—academic departments, funding sources, professional societies, and principal journals—that support the heart of the field.
As just one illustration of the role that researchers in other fields play in the mathematical sciences, the committee examined public data 4 on National Science Foundation NSF grants to get a sense of how much of the research supported by units other than the NSF Division of Mathematical. Sciences DMS has resulted in publications that appeared in journals readily recognized as mathematical science ones or that have a title strongly suggesting mathematical or statistical content.
It also lent credence to the argument that the mathematical sciences research enterprise extends beyond the set of individuals who would traditionally be called mathematical scientists. This exercise revealed the following information:. These publication counts span different ranges of years because the number of publications with apparent mathematical sciences content varies over time, probably due to limited-duration funding initiatives.
For comparison, DMS grants that were active in led to 1, publications. Therefore, while DMS is clearly the dominant NSF supporter of mathematical science research, other divisions contribute in a nontrivial way. Analogously, membership figures from the Society for Industrial and Applied Mathematics SIAM demonstrate that a large number of individuals who are affiliated with academic or industrial departments other than mathematics or statistics nevertheless associate themselves with this mathematical science professional society.
A recent analysis tried to quantify the size of this community on the interfaces of the mathematical sciences. Over the same period, some 75, research papers indexed by Zentralblatt MATH were published by faculty members in other departments of those same 50 universities. The implication is that a good deal of mathematical sciences research—as much as half of the enterprise—takes place outside departments of mathematics. Higher Education 61 6 : This figure shows the fraction of 6, nonstudent members identifying with a particular category.
That analysis also created a Venn diagram, reproduced here as Figure , that is helpful for envisioning how the range of mathematical science research areas map onto an intellectual space that is broader than that covered by most academic mathematics departments. The diagram also shows how the teaching foci of mathematics and nonmathematics departments differ from their research foci.
The tremendous growth in the ways in which the mathematical sciences are being used stretches the mathematical science enterprise—its people, teaching, and research breadth. If our overall research enterprise is operating well, the researchers who traditionally call themselves mathematical scientists—the central ellipse in Figure —are in turn stimulated by the challenges from the frontiers, where new types of phenomena or data stimulate fresh thinking about mathematical and statistical modeling and new technical challenges stimulate deeper questions for the mathematical sciences.
But the cited paper notes that only about 17 percent of the research indexed by Zentralblatt MATH is classified as dealing with statistics, probability, or operations research. FIGURE Representation of the research and teaching span of top mathematics departments and of nonmathematics departments in the same academic institutions. Subjects most published are shown in italics; subjects most taught are underscored.
Higher Education 61 6 , Figure 8. Many people with mathematical sciences training who now work at those frontiers—operations research, computer science, engineering, economics, and so on—have told the committee that they appreciate the grounding provided by their mathematical science backgrounds and that, to them, it is natural and healthy to consider the entire family tree as being a unified whole.
Many mathematical scientists and academic math departments have justifiably focused on core areas, and this is natural in the sense that no other community has a mandate to ensure that the core areas remain strong and robust. But it is essential that there be an easy flow of concepts, results, methods, and people across the entirety of the mathematical sciences. For that reason, it is essential that the mathematical sciences community actively embraces the broad community of researchers who contribute intellectually to the mathematical sciences, including people who are professionally associated with another discipline.
Anecdotal information suggests that the number of graduate students receiving training in both mathematics and another field—from biology to engineering—has increased dramatically in recent years. If this phenomenon is as general as the committee believes it to be, it shows how mathematic sciences graduate education is contributing to science and engineering generally and also how the interest in interfaces is growing.
In order for the community to rationally govern itself, and for funding agencies to properly target their resources, it is necessary to begin gathering data on this trend.
Shop by category
Recommendation The National Science Foundation should systematically gather data on such interactions—for example, by surveying departments in the mathematical sciences for the number of enrollments in graduate courses by students from other disciplines, as well as the number of enrollments of graduate students in the mathematical sciences in courses outside the mathematical sciences. The most effective way to gather these data might be to ask the American Society to extend its annual questionnaires to include such queries.
DMS in particular works to varying degrees with other NSF units, through formal mechanisms such as shared funding programs and informal mechanisms such as program officers redirecting proposals from one division to another, divisions helping one another in identifying reviewers, and so on. Again, for the mathematical sciences community to have a more complete understanding of its reach, and to help funding agencies best target their programs, the committee recommends that a modest amount of data be collected more methodically.
Recommendation The National Science Foundation should assemble data about the degree to which research with a mathematical science character is supported elsewhere in the Foundation. Such an analysis would be of greatest value if it were performed at a level above DMS. A study aimed at developing this insight with respect to statistical sciences within NSF is under way as this is written, at the request of the NSF assistant director for mathematics and physical sciences.
A broader such study would help the mathematical sciences community better understand its current reach, and it could help DMS position its own portfolio to best complement other sources of support for the broader mathematical sciences enterprise.
- Aprils Lady A Novel.
- I giorni in fila (Italian Edition).
- Doppelte Zitate.
- Bad Blood (Rafferty & Llewellyn Book 7).
It would provide a baseline for identifying changes in that enterprise over time. Other agencies and foundations that support the mathematical sciences would benefit from a similar self-evaluation. Data collected in response to Recommendations and can help the community, perhaps through its professional societies, adjust graduate training to better reflect actual student behavior. For example, if a significant fraction of mathematics graduate students take courses outside of mathematics, either out of interest or concern about future opportunities, this is something mathematics departments need to know about and respond to.
Similarly, a junior faculty member in an interdisciplinary field would benefit by knowing which NSF divisions have funded work in their field. While such knowledge can often be found through targeted conversation, seeing the complete picture would be beneficial for individual researchers, and it might alter the way the mathematical sciences community sees itself.
In a discussion with industry leaders recounted in Chapter 5 , the committee was struck by the scale of the demand for workers with mathematical science skills at all degree levels, regardless of their field of training. It heard about the growing demand for people with skills in data analytics, the continuing need for mathematical science skills in the financial sector,. There is a burgeoning job market based on mathematical science skills. However, only a small fraction of the people hired by those industry leaders actually hold degrees in mathematics and statistics; these slots are often filled by individuals with training in computer science, engineering, or physical science.
While those backgrounds appear to be acceptable to employers, this explosion of jobs based on mathematical science skills represents a great opportunity for the mathematical sciences, and it should stimulate the community in three ways:. This is already well recognized in the areas of search technology, financial mathematics, machine learning, and data analytics. No doubt new research challenges will continue to feed back to the mathematical sciences research community as new applications mature;. This will be discussed in the next chapter.
In the past, training in the mathematical sciences was of course essential to the education of researchers in mathematics, statistics, and many fields of science and engineering. And an undergraduate major in mathematics or statistics was always a good basic degree, a stepping-stone to many careers. But the mathematical sciences community tends to view itself as consisting primarily of mathematical science researchers and educators and not extending more broadly.
As more people trained in the mathematical sciences at all levels continue in careers that rely on mathematical sciences research, there is an opportunity for the mathematical sciences community to embrace new classes of professionals. At a number of universities, there are opportunities for undergraduate students to engage in research in nonacademic settings and internship programs for graduate students at national laboratories and industry.
Some opportunities at both the postdoctoral and more senior levels are available at national laboratories. It would be a welcome development for opportunities of this kind to be expanded at all levels. Experiences of this kind at the faculty level can be especially valuable. In an ideal world, the mathematical sciences community would have a clearer understanding of its scale and impacts.
In addition to the steps identified in Recommendations and , annual collection of the following information would allow the community to better understand and improve itself:. Perhaps the mathematical science professional societies, in concert with some funding agencies, could work to build up such an information base, which would help the enterprise move forward.
However, the committee is well aware of the challenges in gathering such data, which would very likely be imprecise and incomplete. Needless to say, these are deeply intertwined, and it is becoming increasingly standard for major research efforts to require expertise in both simulation and large-scale data analysis. Before discussing these two major drivers, it is critical to point out that a great deal of mathematical sciences research continues to be driven by the internal logic of the subject—that is, initiated by individual researchers in response to their best understanding of promising directions.
While over the years there have been important shifts in the level of activity in certain subjects—for example, the growing significance of probabilistic methods, the rise of discrete mathematics, and the growing use of Bayesian statistics—the committee did not attempt to exhaustively survey such changes or prognosticate about the subjects that are most likely to produce relevant breakthroughs. The principal lesson is that it continues to be important for funding sources to support excellence wherever it is found and to continue to support the full range of mathematical sciences research.
Indeed, all areas of the mathematical and statistical sciences have the potential to be important to innovation, but the time scale may be very long, and the nature of the link is likely to be surprising. Many areas of the mathematical and statistical sciences that strike us now as abstract and removed from obvious application will be useful in ways that we cannot currently imagine. On the one hand, we need a research landscape that is flexible and non-prescriptive in terms of areas to be supported. We must have a research funding landscape capable of nurturing a broad range of basic and applied research and that can take into account the changing characteristics of the research enterprise itself.
And on the other hand, we need to build and maintain infrastructure that will connect the mathematical and statistical sciences to strategic growth areas. Quoted text is from p. As science, engineering, government, and business rely increasingly on complex computational simulations, it is inevitable that connections between those sectors and the mathematical sciences are strengthened.
That is because computational modeling is inherently mathematical. Accordingly, those fields depend on—and profit from—advances in the mathematical sciences and the maintainance of a healthy mathematical science enterprise. The same is true to the extent that those sectors increasingly rely on the analysis of large-scale quantities of data.
This is not to say that a mathematical scientist is needed whenever someone builds or exercises a computer simulation or analyzes data although the involvement of a mathematical scientist is often beneficial when the work is novel or complex. But it is true that more and more scientists, engineers, and business people require or benefit from higher-level course work in the mathematical sciences, which strengthens connections between disciplines.
And it is also true that the complexity of phenomena that can now be simulated in silico, and the complexity of analyses made possible by terascale data, are pushing research frontiers in the mathematical sciences and challenging those who could have previously learned the necessary skills as they carry out their primary tasks. As this complexity increases, we are finding more and more occasions where specialized mathematical and statistical experience is required or would be beneficial. Some readers may assume that many of the topics mentioned in this chapter fall in the domain of computer science rather than the mathematical sciences.
In fact, many of these areas of inquiry straddle both fields or could be labeled either way. For example, the process of searching data, whether in a database or on the Internet, requires both the products of computer science research and modeling and analysis tools from the mathematical sciences. The challenges of theoretical computer science itself are in fact quite mathematical, and the fields of scientific computing and machine learning sit squarely at the interface of the mathematical sciences and computer science with insight from the domain of application, in many cases.
Indeed, most modeling, simulation, and analysis is built on the output of both disciplines, and researchers with very similar backgrounds can be found in academic departments of mathematics, statistics, or computer science. There is, of course, a great deal of mathematical sciences research that has not that much in common with computer sciences research—and, likewise, a great deal of computer science research that is not particularly close to the mathematical sciences.
The reason is that mathematical science researchers not only create the tools that are translated into applications elsewhere, but they are also the creative partners who can adapt mathematical sciences results appropriately for different problems. This latter sort of collaboration can result in breakthrough capabilities well worth the investment of time that is sometimes associated with establishing a cross-disciplinary team.
It is not always enough to rely on the mathematics and statistics that is captured in textbooks or software, for two reasons: 1 progress is continually being made, and off-the-shelf techniques are unlikely to be cutting edge, and 2 solutions tailored to particular situations or questions can often be much more effective than more generic approaches. These are the benefits to the nonmathematical sciences members of the team. For mathematical science collaborators, the benefits are likewise dual: 1 fresh challenges are uncovered that may stimulate new results of intrinsic importance to the mathematical sciences and 2 their mathematical science techniques and insights can have wider impact.
In application areas with well-established mathematical models for phenomena of interest—such as physics and engineering—researchers are able to use the great advances in computing and data collection of recent decades to investigate more complex phenomena and undertake more precise analyses.
An Introduction to Mathematics of Emerging Biomedical Imaging
Conversely, where mathematical models are lacking, the growth in computing power and data now allow for computational simulations using alternative models and for empirically generated relationships as means of investigation. Computational simulation now guides researchers in deciding which experiments to perform, how to interpret experimental results, which prototypes to build, which medical treatments might work, and so on.
Indeed, the ability to simulate a phenomenon is often regarded as a test of our ability to understand it. Over the past years, computational capabilities reached a threshold at which statistical methods such as Markov chain Monte Carlo methods and large-scale data mining and analysis became feasible, and these methods have proved to be of great value in a wide range of situations.
For example, at one of its meetings the study committee saw a simulation of biochemical activity developed by Terrence Sejnowski of the Salk Institute for Biological Studies. It was a tour de force of computational simulation—based on cutting-edge mathematical sciences and computer science—that would not have been feasible until recently and that enables novel investigations into complex biological phenomena. As another example, over the past 30 years or so, ultrasound has progressed from providing still images to dynamically showing a beating heart and, more recently, to showing the evolution of a full baby in the womb.
The mathematical basis for ultrasound requires solving inverse problems and draws. As ultrasound technologies have improved, new mathematical challenges also need to be addressed. The mathematical sciences contribute in essential ways to all the items on this list except the fourth. The great majority of computational science and engineering can be carried out well by investigators from the field of study: They know how to create a mathematical model of the phenomenon under study, and standard numerical solution tools are adequate.
However, as the phenomena being modeled become increasingly complex, perhaps requiring specialized articulation between models at different scales and of different mathematical types, specialized mathematical science skills become more and more important. Absent such skills and experience, computational models can be unstable or even produce unreliable results.
Validation of such complex models requires very specialized experience, and the critical task of quantifying their uncertainties can be. The research teams must have strong statistical skills in order to create reliable knowledge in such cases. In response to the need to harness this vast computational power, the community of mathematical scientists who are experts in scientific computation continues to expand. This cadre of researchers develops improved solution methods, algorithms for gridding schemes and computational graphics, and so on. Much more of that work will be stimulated by new computer architectures now emerging.
And, a much broader range of mathematical science challenges stem from this trend.
Shop with confidence
The theory of differential equations, for example, is challenged to provide structures that enable us to analyze approximations to multiscale models; stronger methods of model validation are needed; algorithms need to be developed and characterized; theoretical questions in computer science need resolution; and so on.
High-throughput data in biology has been an important driver for new statistical research over the past years. Research in genomics and proteomics relies heavily on the mathematical sciences, often in challenging ways, and of disease, evolution, agriculture, and other topics have consequently become quantitative as genomic and proteomic information is incorporated as a foundation for research.
Arguably, this development has placed statisticians as central players in one of the hottest fields of science. Over the next years, acquiring genomic data will become fairly straightforward, and increasingly it will be available to illuminate biological processes.
Biomedical Computation Review , September 1, gives an overview of sources of error and points to some striking published studies. See also National Research Council, As biology transitions from a descriptive science to a quantitative science, the mathematical sciences will play an enormous role.
To different degrees, the social sciences are also embracing the tools of the mathematical sciences, especially statistics, data analytics, and mathematics embedded in simulations. For example, statistical models of disease transmission have provided very valuable insights about patterns and pathways. Business, especially finance and marketing, is increasingly dependent on methods from the mathematical sciences. Some topics in the humanities have also benefited from mathematical science methods, primarily data mining, data analysis, and the emerging science of networks.
The mathematical sciences are increasingly contributing to data-driven decision making in health care. Operations research is being applied to model the processes of health care delivery so they can be methodically improved. The applications use different forms of simulation, discrete optimization, Markov decision processes, dynamic programming, network modeling, and stochastic control.
As health care practices move to electronic health care records, enormous amounts of data are becoming available and in need of analysis; new methods are needed because these data are not the result of controlled trials. The new field of comparative effectiveness research, which relies a great deal on statistics, aims to build on data of that sort to characterize the effectiveness of various medical interventions and their value to particular classes of patients. Embedded in several places in this discussion is the observation that data volumes are exploding, placing commensurate demands on the mathematical sciences.
This prospect has been mentioned in a large number of previous reports about the discipline, but it has become very real in the past 15 years or so. What really matters is our ability to derive from them new insights, to recognize relationships, to make increasingly accurate predictions. Our ability, that is, to move from data, to knowledge, to action. Large, complex data sets and data streams play a significant role in stimulating new research applications across the mathematical sciences, and mathematical science advances are necessary to exploit the value in these data.
However, the role of the mathematical sciences in this area is not always recognized. Indeed, the stated goals for the OSTP initiative,. Multiple issues of fundamental methodology arise in the context of large data sets. Some arise from the basic issue of scalability—that techniques developed for small or moderate-sized data sets do not translate to modern massive data sets—or from problems of data streaming, where the data set is changing while the analysis goes on. Data that are high-dimensional pose new challenges: New paradigms of statistical inference arise from the exploratory nature of understanding large complex data sets, and issues arise of how best to model the processes by which large, complex data sets are formed.
Not all data are numerical—some are categorical, some are qualitative, and so on—and mathematical scientists contribute perspectives and techniques for dealing with both numerical and non-numerical data, and with their uncertainties. Noise in the data-gathering process needs to be modeled and then—where possible—minimized; a new algorithm can be as powerful an enhancement to resolution as a new instrument.
Often, the data that can be measured are not the data that one ultimately wants. This results in what is known as an inverse problem—the process of collecting data has imposed a very complicated transformation on the data one wants, and a computational algorithm is needed to invert the process.
The classic example is radar, where the shape of an object is reconstructed from how radio waves bounce off it. Simplifying the data so as to find its underlying structure is usually essential in large data sets. The general goal of dimensionality reduction—taking data with a large number of measurements and finding which combinations of the measurements are. Various methods with their roots in linear algebra and statistics are used and continually being improved, and increasingly deep results from real analysis and probabilistic methods—such as random projections and diffusion geometry—are being brought to bear.
Statisticians contribute a long history of experience in dealing with the intricacies of real-world data—how to detect when something is going wrong with the data-gathering process, how to distinguish between outliers that are important and outliers that come from measurement error, how to design the data-gathering process so as to maximize the value of the data collected, how to cleanse the data of inevitable errors and gaps. As data sets grow into the terabyte and petabyte range, existing statistical tools may no longer suffice, and continuing innovation is necessary.
In the realm of massive data, long-standing paradigms can break—for example, false positives can become the norm rather than the exception—and more research endeavors need strong statistical expertise. For example, in a large portion of data-intensive problems, observations are abundant and the challenge is not so much how to avoid being deceived by a small sample size as to be able to detect relevant patterns. In that approach, one uses a sample of the data to discover relationships between a quantity of interest and explanatory variables.
Strong mathematical scientists who work in this area combine best practices in data modeling, uncertainty management, and statistics, with insight about the application area and the computing implementation. These prediction problems arise everywhere: in finance and medicine, of course, but they are also crucial to the modern economy so much so that businesses like Netflix, Google, and Facebook rely on progress in this area.
A recent trend is that statistics graduate students who in the past often ended up in pharmaceutical companies, where they would design clinical trials, are increasingly also being recruited by companies focused on Internet commerce. Finding what one is looking for in a vast sea of data depends on search algorithms. This is an expanding subject, because these algorithms need to search a database where the data may include words, numbers, images and video, sounds, answers to questionnaires, and other types of data, all linked.
New York Times , August 5. New techniques of machine learning continue to be developed to address this need. Another new consideration is that data often come in the form of a network; performing mathematical and statistical analyses on networks requires new methods. Statistical decision theory is the branch of statistics specifically devoted to using data to enable optimal decisions. What it adds to classical statistics beyond inference of probabilities is that it integrates into the decision information about costs and the value of various outcomes.
Ideas from statistics, theoretical computer science, and mathematics have provided a growing arsenal of methods for machine learning and statistical learning theory: principal component analysis, nearest neighbor techniques, support vector machines, Bayesian and sensor networks, regularized learning, reinforcement learning, sparse estimation, neural networks, kernel methods, tree-based methods, the bootstrap, boosting, association rules, hidden Markov models, and independent component analysis—and the list keeps growing.
This is a field where new ideas are introduced in rapid-fire succession, where the effectiveness of new methods often is markedly greater than existing ones, and where new classes of problems appear frequently. Large data sets require a high level of computational sophistication because operations that are easy at a small scale—such as moving data between machines or in and out of storage, visualizing the data, or displaying results—can all require substantial algorithmic ingenuity.
As a data set becomes increasingly massive, it may be infeasible to gather it in one place and analyze it as a whole. Thus, there may be a need for algorithms that operate in a distributed fashion, analyzing subsets of the data and aggregating those results to understand the complete set.