Martin L. Puterman – Publications
Updated October 9, 2011
BOOKS
Puterman, M.L., (Ed.) Dynamic Programming and Its Applications, Academic Press, 1978. (Proceedings of The International Conference on Dynamic Programming, Vancouver, 1978)
Puterman, M. L., Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley and Sons, New York, NY, 1994, 649 pages. (Winner 1995 INFORMS Lanchester Prize for best operations research publication; paperback version released, March 2005.)
JOURNAL ARTICLES
1. Puterman, M.L., “Sensitive Discount Optimality in Controlled One‑Dimensional Diffusions,” Annals of Probability, 2,3:408‑419 (1974).
2. Puterman, M.L., “A Diffusion Process Model for a Storage System,” Logistics, Volume 1, North Holland/TIMS Studies in the Management Sciences (M.A. Geisler, Ed.) 1:143‑159 (1975). (Invited paper)
3. Puterman, M.L., “Optimal Control of Diffusion Processes with Reflection, “Journal of Optimization Theory and Its Applications, 22, 1:103‑115, 1977.
4. Puterman, M.L. and M.C. Shin, “Modified Policy Iteration Algorithms for Discounted Markov Decision Problems,” Management Science, 24, 11:1127‑1137, 1978.
5. Everitt,R., M.L. Puterman, N. Sonntag, and P. Whalen, “A Mathematical Programming Model for the Management of a Renewable Resource System: The Kemano II Development Project,” Journal of Fisheries Research Board of Canada, 35, 2:235‑246, 1978.
6. Puterman, M.L. and S.L. Brumelle, “On the Convergence of Policy Iteration in Stationary Dynamic Programming,” Mathematics of Operations Research, 4,1:60‑69, 1979.
7. Puterman, M.L., “On the Convergence of Policy Iteration for Controlled Diffusions,” Journal of Optimization Theory and Its Applications, 33, 1:137‑144, 1981.
8. Puterman, M.L. and M.C. Shin, “Action Elimination Procedures for Modified Policy Iteration Algorithms,” Operations Research, 30, 2:301‑318, 1982.
9. van Nunen, J.A.E.E., and M.L. Puterman, “On Computing Optimal Policies for G/M/s Queueing Systems,” Management Science, 29, 6: 725‑734, 1983.
10. Speert, D.P., Eftekhar, F. and M.L. Puterman, “Non‑Opsonic Phagocytosis of Pseudomonas Aeruginosa Strains from Cystic Fibrosis Patients,” Infection and Immunity, 43, 3; 1006‑1011, 1984.
11. Schweitzer, P.J., M.L. Puterman, and K.W. Kindle, “Iterative Aggregation ‑ Disaggregation Procedures for Solving Discounted Semi‑Markovian Reward Processes,” Operations Research, 35, 3, 589‑605, 1985.
12. Gosling, B.J. and M.L. Puterman, “Ridge Estimation in Regression Problems with Autocorrelated Errors: A Monte Carlo Study,” Communications in Statistics, 14, 3, 577‑614, 1985.
13. Kastrukoff, L.F., A.S. Lau, and M.L. Puterman, “Genetics of Natural Resistance to Herpes Simplex Virus 1: Latent Infection of the Peripheral Nervous System in Mice,” The Journal of General Virology, 67, 613‑631, 1986.
14. Sandor, G., M.L. Puterman, M.W. Patterson, M.A. Tipple and D.J. Vince, “The Effect of Pressure Loading, Volume Loading and Cardiomyopathy on Left Ventricular Diastolic Operant Chamber Stiffness and Myocardial Stiffness in Congenital Heart Disease, With Re‑evaluation of Normal Paediatric Values,” Journal of the American College of Cardiology, 8, 2, 371‑378, 1986.
15. Junker, A., E.A. Clark, H.D. Ochs, M.L. Puterman and R.J. Wedgewood, “Abnormal Antibody Responses to Bacteriophage X174 in Patients with Acute Epstein‑Barr Virus Infection” Journal of Immunology and Immunopathology, 40, 436‑446, 1986.
16. Hancock, E.J., K. Pot, M.L. Puterman and A.J. Tingle, “Dissociation Between Rubella HAI Titers and Whole Virus ELISA Values in Congenital Rubella Syndrome”, The Journal of Infectious Diseases, 154, 1031‑1033, 1986.
17. Petty, R.E., D.W.C. Hunt, D.F. Rollins, M.L. Schroeder and M.L. Puterman, “Immunity to Soluble Retinal Antigen in Patients with Uveitis Accompanying Juvenile Rheumatoid Arthritis,” Arthritis and Rheumatism, 30, 287‑29, 1987.
18. Hordijk, A. and M.L. Puterman, “On the Convergence of Policy Iteration in Finite State Average Reward Markov Decision Processes: The Unichain Case,” Mathematics of Operations Research, 12, 163‑176, 1987.
19. Sandor, G., P. Schumacher, M.L. Puterman and R. Popov, “Normal Serial Variability for M‑Mode and Digitized Echocardiography in a Paediatric Population,” Paediatric Cardiology, 8, 23‑26, 1987.
20. Puterman, M.L. and L.C. Thomas “A Note on Computing Optimal Policies for G/M/s Queueing Systems’,” Management Science, 33, 939‑943, 1987.
21. Lokitch, G., V. Singh, M.L. Puterman, W.J. Godolphin, S. Sheps, A.J. Tingle, F.Wong and G. Quigley, “Age‑Related Changes in Humoral and Cell‑Mediated Immunity in Down Syndrome Children Living at Home’, Pediatric Research, 22, 536‑540, 1987.
22. van Dijk, N. and M.L. Puterman, “Perturbation Theory for Markov Reward Processes with Applications to Queueing Systems,” Advances in Applied Probability, 20, 79‑98, 1988.
23. Puterman, M.L., “Leverage and Influence in Autocorrelated Regression Models,” Applied Statistics, 37, 76‑88, 1988.
24. Puterman, M.L., N. Hrboticky and S. Innis, “Nonlinear Estimation of Parameters in Biphasic Arrhenius Plots,” Analytical Biochemistry, 170, 409‑420, 1988.
25. Thomas, E.E., M.L. Puterman, E. Kawano and M. Curran, “Evaluation of Immunoassays for the Detection of Rotavirus in Stool Samples from a Pediatric Population,” Journal of Clinical Microbiology, 26, 1189‑1193, 1988.
26. Magil, A.B., M.L. Puterman, M.S. Ballow, V. Chan, D.S. Lirenman, A. Rae and R.A.L. Sutton, “Prognostic Factors in Diffuse Proliferative Lupus Glomerulonehritus,” Kidney International, 34, 511‑517, 1988.
27. Lamond, B. and M.L. Puterman, “Generalized Inverses in Discrete Time Markov Decision Processes” SIAM Journal of Matrix Analysis and Applications, 10, 118‑134, 1989.
28. Lokitch, G., M.L. Puterman, W.J. Godolphin, S. Sheps, A.J. Tingle, and G. Quigley, “Infection and Immunity in Down Syndrome: A Trial of Long Term Low Dose Oral Zinc,” Journal of Pediatrics, 114, 781‑787, 1989.
29. Gribble, M.J., M.L. Puterman and N.M. McCallum, “Pyuria: Its Relationship to Bacteria in Spinal Cord Injured Patients on Intermittent Catheterization” Archives of Physical Medicine and Rehabilitation, 70, 376‑379, 1989.
30. Chanson, S., M.L. Puterman and W. Wong, “A Markov Decision Process Model for Computer System Load Control”, INFOR, 27, 387‑402, 1989.
31. Mackinnon, M.J. and M.L. Puterman, “Collinearity in Generalized Linear Models,” Communications in Statistics, 18(9), 3463-3472, 1989.
32. Hrboticky, N., MacKinnon, M.J., Puterman, M.L. and Innis, S.M. “Effect of a linoleic acid rich vegetable oil “infant” formula on brain synaptosomal lipid accretion and enzyme thermotropic behavior in the newborn piglet”. J. Lipid Research, 30, 1173-1184, 1989.
33. Tingle, A.J., K.H. Pot, F.P. Yong, M.L. Puterman and E.J. Hancock. “Kinetics of Isotype-Specific Humoral Immunity in Rubella Vaccine-Associated Arthropathy”. Clinical Immunology and Immunopathology, 53, S99-S106, 1989.
34. Rotschild, A., A. Solimano, M.L. Puterman, J. Smith, A. Sharma, and S. Albersheim, “Large Magnitude Compliance Increase in Response to ‑2 Agonist in Premature Infants Developing Bronchopulmonary Dysplasia,” Journal of Pediatrics, 115, 984-991, 1989.
35. Rotschild, A., E. Ling, M.L. Puterman and D. Farquharson, “Neonatal Outcome of Prolonged Rupture of Membranes,” American Journal of Obstetrics and Gynaecology, 162, 46-52, 1989.
36. Puterman, M.L., P. Schumacher and G.S.S. Sandor, “Optimal Choice of Prognostic Variables with an Application to Cardiac Monitoring Using M‑Mode Echocardiography,” Statistics in Medicine, 9, 273-286, 1990.
37. Haviv, M. and Puterman, M.L. “An Improved Algorithm for Solving Communicating Average Reward Markov Decision Processes” Annals of Operations Research, (special issue on Markov Decision Processes) 28, 229-242, 1991.
38. Rotschild, A., Chitiyat, D., Puterman, M.L., Phang, M.S., Ling, E., Baldwin, V., “Optimal Positioning of Endotracheal Tubes For Ventilation of Preterm Infants”, American Journal of Diseases of Children, 145, 1007-1012, 1991.
39. Sandor, G.S., Puterman, M.L., P.Rogers, K.W. Chan, S. Pritchard, R. Popov, “Early Prediction of Anthracycline Cardionyapthy using Standard M-Mode and Digitized Echocardiography”, American Journal of Pediatric Hematology/ Oncology, 14, 151-157, 1992.
40. Leroux, B.G. and Puterman, M.L., “Maximum Penalized Likelihood Estimation for Independent and Markov-Dependent Poisson Mixtures”, Biometrics, 48, 545-558, 1992.
41. Le, N.D., Leroux, B.L. and Puterman, M.L., “Exact Likelihood Evaluation in a Markov Mixture Model for Time Series of Seizure Counts” Biometrics, 48, 317-322, 1992.
42. Haviv, M. and Puterman, M.L., “Estimating the Value of a Discounted Reward Process”, Operations Research Letters, 11 267-272, 1992.
43. Massoud, E.A., Sekhon, H.S., Rotschild, A., Puterman, M.L., Matsui, R. and Thurlbeck, W.M., “In Vitro Branching Morphogenesis of the Fetal Rat Lung”, Pediatric Pulmonology 15, 89-97, 1993.
44. Gribble, M.J. and Puterman, M.L., “Prophylaxis of Urinary Tract Infection in Persons with Recent Spinal Cord Injury: A Prospective, Randomized, Double-Blind, Placebo-Controlled Study of Trimethoprim-Sulfamethoxazole”, American Journal of Medicine, 95, 151- 162, 1993.
45. Speert, D.P., Puterman, M.L. et. al. “A Multicenter Comparison of Methods for Typing Strains of Pseudomona Aeruginosa predominantly from Patients with Cystic Fibrosis”, Journal of Infectious Diseases, 169, 134-142, 1994.
46. Leroux, B. and Puterman, M.L. “Reponse to: Empirical Fitting of Discrete Distributions” , Biometrics, 50, 578-579, 1994.
47. Wang, P., Puterman, M.L., Le, N. and Cockburn, I. “Mixed Poisson Regression Models with Covariate Dependent Rates”, Biometrics, 52, 381-400, 1996.
48. Rotschild, A., Massoud, E.A., Puterman, M.L., Sekhon, H.S. and Thurlbeck, W.M., “Development of the Pulmonary Airways in the Fetal Rat and its Relationship to Prenatal Environment”, Pediatric Pulmonogy, 21, 219-226, 1996.
49. Cherukupalli, K., Larson, J.E., Puterman, M.L. “Comparative Biochemistry of Gestational and Postnatal Lung Growth and Development in the Rat and Human”, Pediatric Pulmonology, 24, 12-21, 1997.
50. Dear K., Dobson, A. and Puterman, ML, “Estimated Correlations from Epidemiological Data in the Presence of Measurement Error”, Statistics in Medicine, 16, 2177-2189, 1997.
51. Wang, P., Cockburn, I. and Puterman, M.L.,. “Analysis of Patent Data – A Mixed Poisson Regression Model Approach”, Journal of Business and Economic Statistics, 16, 27-41, 1998.
52. Haviv, M. and Puterman, M.L. “Bias Optimality in Controlled Queueing Systems”, Journal of Applied Probability, 35, 136-150, 1998.
53. Volk, T., Hahn, L., Hayden, R. Abel, J., Puterman, M.L. and Tyers, G.F. Reliability Audit of a Regional Cardiac Registry, Journal of Thoracic and Cardiovascular Surgery, 903-910, 1998.
54. Wang, P. and Puterman, ML “Mixed Logistic Regression Models”, Journal of Agricultural, Biological and Environmental Statistics, 3, 175-200, 1998.
55. Wang, P. and Puterman, M.L., “Markov Poisson Regression Models for Discrete Time Series; Parts I and II”, Journal of Applied Statistics, 26, 855-869, 871-882, 1999.
56. Kapalka, B, Katiriciogulu, K. and Puterman M.L. ” Inventory Control in a Retail Environment with Lost Sales and Service Level Constraints”, Production and Operations Management, 8, 393-408, 2000.
57. Lewis, M. and Puterman, M.L., ” A Note on Bias Optimality in Controlled Queueing Systems”, Journal of Applied Probability, 37, 1, 300-305, 2000.
58. Lewis, M. and Puterman, M.L., “A Probabilistic Analysis of Bias Optimality in Unichain Markov Decision Processes” IEEE Automatic Control, 46, 1, 96-100, 2001
59. Swami, S., Weinberg, C. and Puterman, M.L. “Play it Again Sam, Optimal Replacement Policies for a Motion Picture Exhibitor”, Manufacturing and Service Operations Management, 3, 369-386, 2001.
60. Wang, P. and Puterman, M.L. “ Analysis of Longitudinal Data of Epileptic Seizure Counts – A Two-State Hidden Markov Regression Approach” Biometrical Journal, 43, 8, 941-962, 2001.
61. Ding, X., Puterman, M.L. and Bisi, A., “The Censored Newsvendor and the Optimal Acquisition of Information” Operations Research, 50, 3, 517-527, 2002
62. Urbanovich, E., Young, E., Puterman, M.L. and Fattedad, S. “Identifying High-risk Workers Compensation Claims using Logistic Regression” Interfaces, 33, 3, 15-26, 2003.
63. Begen, M.A. and Puterman, M.L., ”Development of a Catch Allocation Tool Design for Production Planning at JS McMillan Fisheries”, INFOR, 41,3, 235-244, 2003.
64. Goto, J., Lewis, M.E. and Puterman, M.L., “Coffee, Tea or …? A Markov Decision Process Model for Airline Meal Provisioning”, Transportation Science, 38,1, 107-118, 2004.
65. Carvalho, A. and Puterman, M.L. “Learning and pricing in an Internet environment with binomial demands”, Journal of Revenue and Pricing, 3, 4, 320-336, 2005.
66. Chen, L., Gerschman, M., Odegaard, F., Puterman, D.K., Puterman, M.L., Quee, R. , Designing an Efficient Hospital Porter System, Health Care Quarterly 2005.
67. Odegaard, F., Chen, L., Quee, R., Puterman, M. Improving the Efficiency of Hospital Porter Services, Part 1: Study Objectives and Results. Journal for Healthcare Quality: 29, 4-11 ,2007.
68. Odegaard, F., Chen, L., Quee, R., Puterman, M. Improving the Efficiency of Hospital Porter Services, Part 2: Schedule Optimization and Simulation Model Study Objectives and Results. Journal for Healthcare Quality:29 12-18, 2007
69. Patrick, J., and Puterman, M. Improving Resource Utilization for Diagnostic Services Through Flexible Inpatient Scheduling. Journal of the Operational Research Society 58, 235–245, 2007.
70. Patrick, J. and Puterman, M. Reducing Wait Times Through Operations Research: Optimizing the Use of Surge Capacity, Healthcare Policy 3, 75-88, 2008.
71. Grassmann, W., M. Puterman, P. L’Ecuyer, A. Ingolfsson, “Four Canadians Contributions to Stochastic Modeling”, INFOR special issue commemorating the 50th Anniversary of the Canadian Operations Research Society, 4, 6, 3-14, 2008.
72. Patrick, J., Puterman, M.L. and Queyranne, M. “Dynamic Multi-Priority Patient Scheduling” Operations Research, 56, 1507-152, 2008
73. Puterman, ML and Wittman, S. “Match Play; A Statistical Analysis of PGA Tour Players’ Career Performance” Quantitative Analysis in Sports, 5, issue 1, article 11, 2008.
74. Lavieri, M., Regan, S., Ratner, P. and Puterman, M, “Applying Operations Research to Health Human Resource Planning: A Case Study of Registered Nurse Workforce Planning in British Columbia”, Healthcare Policy, 4, 2008.
75.Lavieri, M. and Puterman, M., “Optimal Nurse Workforce Planning” Health Care Management Science, 12,2, 2009.
76.Santibáñez, P., Chow, V., French, J., Puterman, M. and Tyldesley, S. “Reducing patient wait times and improving resource utilization at British Columbia Cancer Agency’s ambulatory care unit through simulation”, Health Care Management Science, 12, 4, 392-407, 2009
77. Chow, V., M. Puterman, N. Salaherad, K. Huang, D. Atkins, “Reducing Surgical Ward Congestion”, POMS, Special Issue on Health Care Management, 20, 3, 418-430, 2011.
78. Puterman, M. L. and. Wang, Q, “Optimal Dynamic Clustering Through Relegation and Promotion: How to Design a Competitive Sports League” Quantitative Analysis in Sports, 7, issue 2, Article 7, 2011.
79. Odegaard, F and Puterman, M.L, “Selling in Channels with Uncertain Price”, Journal of Revenue Management, Accepted July 2011.
80. Odegaard, F and Puterman, M.L. “Estimating Intermediate Price Transitions in Online Auctions”, Applied Stochastic Models in Business and Industry, Accepted August 2011.
81. Zhang, Y., M. Puterman, M. Nelson and D. Atkins, “Simulation Optimization Approach to Long Term Care Capacity Planning” , Operations Research, Accepted August 2011.
82. Zhang, Y., M. Puterman and D. Atkins, “Analytical Long-Term Care Capacity Planning: The Shortcomings of Ratio Based Forecasts” Health Care Policy, Accepted October 2011.
PAPERS UNDER REVIEW
Lavieri, M., M. Puterman, S. Tyldesley and W. Morris, “When to Treat Prostate Cancer Patients Based on their PSA Dynamics”
Ghassemi, F., S. Shechter, M. Puterman, C. Nguan, “Optimal Search with Application to Minimally Invasive Surgery”,
Puterman, M.L, Y. Zhang, S. Aydede, H. Bavafa, S. MacLeod, and J. Mackenzie, “A Framework for Evaluating a Lean Health Care Program”
Saure, A., J. Patrick and M. Puterman “Optimal Multi-Appointment Scheduling”,
CONFERENCE PROCEEDINGS (REFEREED)
Puterman, M.L. and S.L. Brumelle, “The Analytic Theory of Policy Iteration,” Dynamic Programming and Its Applications, M.L. Puterman, Ed., Academic Press, 91‑114, 1981.
Brumelle, S.L. and M.L. Puterman, “Newton’s Method for W‑Convex Operators,” Generalized Concavity in Optimization and Economics, S. Schaible and W. Ziemba, (Eds.) Academic Press, 399‑414, 1981.
Carvalho, A. and M.L. Puterman, “Optimal Pricing and Reinforcement Learning”, IJCNN, 2003.
CONFERENCE PROCEEDINGS (NOT REFEREED)
Puterman, M.L., “Computational Methods for Markov Decision Processes,” The Proceedings of the JACC Conference, Charlottesville, Virginia 1981. (Invited Paper).
van Nunen, J.A.E.E. and M.L. Puterman, “On Solving G/M/s Queueing Control Systems,” Proceedings of the German Operations Research Society, 1981.
Jorion, P. and M.L. Puterman, “An Application of Ridge Regression to Exchange Rate Determination,” 1984 Business and Economic Statistics Sections, Proceedings of American Statistical Association.
Puterman, M.L., “Leverage and Influence in Autocorrelated Regression Models,” 1985 Business and Economic Statistics Section, Proceedings of the American Statistical Association.
Boutlier, C. and Puterman, M.L. “Process Oriented Planning and Average Reward Optimality” IJCAI Conference, 1995.
BOOK REVIEWS
van Nunen, J.A.E.E., “Contracting Dynamic Programming,” Journal of the American Statistical Association, 73, 362:446 (1978).
Ladany, S.P. and R.E. Machol, “Optimal Strategies in Sports,” Journal of the American Statistical Association, 74, 367: 739‑740 (1979).
Moder, J.J. and Elmaghraby, Handbook of Operations Research (Vols. 1 and 2), Journal of the American Statistical Association, 75, 372: 1033‑1035, (1980).
Weeda, P.J., “Finite Generalized Markov Programming,” Journal of American Statistical Association, 77, 378:489 (1982).
van der Duyn Schouten, F., “Markov Decision Processes with Continuous Time Parameter,” Journal of the American Statistical Association, 80, 390:491(1985).
CHAPTERS IN BOOKS
Puterman, M.L. and H.G. Dunn, “Statistical Analysis of Mild Brain Dysfunction in Children of Low Birth Weight: Comparison with Control Subjects of Full Birth Weight and Subdivision by Cluster Analysis,” Chapter 8 in Sequelae of Low Birthweight: The Vancouver Study, H.G. Dunn, ed., MacKeith Press, 1986.
Puterman, M.L., “Dynamic Programming” in Encyclopedia of Physical Science and Technology, R.A. Meyers, (ed.), Academic Press, Volume 4, 438‑463, 1987.
Puterman, M.L., “Markov Decision Processes” in Handbook of Operations Research, Volume II, D.Heyman and M. Sobel, eds., North Holland Publishers, 331-434, 1990.
Puterman, M.L. and L. Kirby, “Probability and Statistics” in L. Kirby, DNA Fingerprinting: An Introduction, MacMillan, 149-177, 1990.
Lewis, M. and Puterman, M.L. “Bias Optimality”, Handbook of Markov Decision Processes; Methods and Applications, E. Feinberg and A. Schwartz (Eds.) 2002, 89-112.
CASES
Whistler-Blackcomb (A) and (B).
MAGAZINE ARTICLES
Atkins, D.A., Begen, M.A., Kluczny, B., Parkinson, A. and Puterman, M. L., “Right on Queue; OR models improve passenger flows and customer service at Vancouver International Airport”, OR-MS Today, April 2003.