How Open Source Engineering Could Be Used To Recruit Top International Engineers Literature Review\Estimating development effort in Free\Open source software projects by mining software repositories: a case study of OpenStack

Estimating development effort in Free/Open source software projects by mining software repositories: a case study of OpenStack[edit | edit source]

Robles, G., González-Barahona, J. M., Cervigón, C., Capiluppi, A., & Izquierdo-Cortázar, D. (2014). Estimating development effort in Free/Open source software projects by mining software repositories: a case study of OpenStack. Proceedings of the 11th Working Conference on Mining Software Repositories, 222–231. https://doi.org/10.1145/2597073.2597107

• Abstract: Because of the distributed and collaborative nature of free / open source software (FOSS) projects, the development effort invested in a project is usually unknown, even after the software has been released. However, this information is becoming of major interest, especially ---but not only--- because of the growth in the number of companies for which FOSS has become relevant for their business strategy. In this paper we present a novel approach to estimate effort by considering data from source code management repositories. We apply our model to the OpenStack project, a FOSS project with more than 1,000 authors, in which several tens of companies cooperate. Based on data from its repositories and together with the input from a survey answered by more than 100 developers, we show that the model offers a simple, but sound way of obtaining software development estimations with bounded margins of error.
• Identifying full time contributor (P2)
• Rest contributor has small impact (P2)
• Two possible measures to identify full time contributor (P2)
• OpenStack

Effort estimation of FLOSS projects: a study of the Linux kernel. Empirical Software Engineering[edit | edit source]

Capiluppi, A., & Izquierdo-Cortázar, D. (2013). Effort estimation of FLOSS projects: a study of the Linux kernel. Empirical Software Engineering, 18(1), 60–88. https://doi.org/10.1007/s10664-011-9191-7

• Abstract: Empirical research on Free/Libre/Open Source Software (FLOSS) has shown that developers tend to cluster around two main roles: "core" contributors differ from "peripheral" developers in terms of a larger number of responsibilities and a higher productivity pattern. A further, cross-cutting characterization of developers could be achieved by associating developers with "time slots", and different patterns of activity and effort could be associated to such slots. Such analysis, if replicated, could be used not only to compare different FLOSS communities, and to evaluate their stability and maturity, but also to determine within projects, how the effort is distributed in a given period, and to estimate future needs with respect to key points in the software life-cycle (e.g., major releases). This study analyses the activity patterns within the Linux kernel project, at first focusing on the overall distribution of effort and activity within weeks and days; then, dividing each day into three 8-hour time slots, and focusing on effort and activity around major releases. Such analyses have the objective of evaluating effort, productivity and types of activity globally and around major releases. They enable a comparison of these releases and patterns of effort and activities with traditional software products and processes, and in turn, the identification of company-driven projects (i.e., working mainly during office hours) among FLOSS endeavors. The results of this research show that, overall, the effort within the Linux kernel community is constant (albeit at different levels) throughout the week, signalling the need of updated estimation models, different from those used in traditional 9am–5pm, Monday to Friday commercial companies. It also becomes evident that the activity before a release is vastly different from after a release, and that the changes show an increase in code complexity in specific time slots (notably in the late night hours), which will later require additional maintenance efforts.

Effort estimation by characterizing developer activity[edit | edit source]

Amor, J. J., Robles, G., & Gonzalez-Barahona, J. M. (2006). Effort estimation by characterizing developer activity. Proceedings of the 2006 International Workshop on Economics Driven Software Engineering Research, 3–6. https://doi.org/10.1145/1139113.1139116

• Abstract: During the latest years libre (free, open source) software has gained a lot of attention from the industry. Following this interest, the research community is also studying it. For instance, many teams are performing quantitative analysis on the large quantity of data which is publicly available from the development repositories maintained by libre software projects. However, not much of this research is focused on cost or effort estimations, despite its importance (for instance, for companies developing libre software or collaborating with libre software projects), and the availability of some data which could be useful for this purpose. Our position is that classical effort estimation models can be improved from the study of these data, at least when applied to libre software. In this paper, we focus on the characterization of developer activity, which we argue can improve effort estimation. This activity can be traced with a lot of detail, and the resulting data can also be used for validation of any effort estimation model.
• Better estimation than size of code (P1)

A Systematic Review of Software Development Cost Estimation Studies[edit | edit source]

Jorgensen, M., & Shepperd, M. (2007). A Systematic Review of Software Development Cost Estimation Studies. IEEE Transactions on Software Engineering, 33(1), 33–53. https://doi.org/10.1109/TSE.2007.256943

• Abstract: This paper aims to provide a basis for the improvement of software-estimation research through a systematic review of previous work. The review identifies 304 software cost estimation papers in 76 journals and classifies the papers according to research topic, estimation approach, research approach, study context and data set. A Web-based library of these cost estimation papers is provided to ease the identification of relevant estimation research results. The review results combined with other knowledge provide support for recommendations for future software cost estimation research, including: 1) increase the breadth of the search for relevant studies, 2) search manually for relevant papers within a carefully selected set of journals when completeness is essential, 3) conduct more studies on estimation methods commonly used by the software industry, and 4) increase the awareness of how properties of the data sets impact the results when evaluating estimation methods

How to estimate the contribution of an individual to a software project?[edit | edit source]

How to estimate the contribution of an individual to a software project? (n.d.). Stack Overflow. Retrieved February 26, 2021, from https://stackoverflow.com/questions/2437881/how-to-estimate-the-contribution-of-an-individual-to-a-software-project

An empirical validation of software cost estimation models[edit | edit source]

Kemerer, C. F. (1987). An empirical validation of software cost estimation models. Communications of the ACM, 30(5), 416–429. https://doi.org/10.1145/22899.22906

• Abstract: Practitioners have expressed concern over their inability to accurately estimate costs associated with software development. This concern has become even more pressing as costs associated with development continue to increase. As a result, considerable research attention is now directed at gaining a better understanding of the software-development process as well as constructing and evaluating software cost estimating tools. This paper evaluates four of the most popular algorithmic models used to estimate software costs (SLIM, COCOMO, Function Points, and ESTIMACS). Data on 15 large completed business data-processing projects were collected and used to test the accuracy of the models' ex post effort estimation. One important result was that Albrecht's Function Points effort estimation model was validated by the independent data provided in this study [3]. The models not developed in business data-processing environments showed significant need for calibration. As models of the software-development process, all of the models tested failed to sufficiently reflect the underlying factors affecting productivity. Further research will be required to develop understanding in this area.

Cost models for future software life cycle processes: COCOMO 2.0. Annals of Software Engineering[edit | edit source]

Boehm, B., Clark, B., Horowitz, E., Westland, C., Madachy, R., & Selby, R. (1995). Cost models for future software life cycle processes: COCOMO 2.0. Annals of Software Engineering, 1(1), 57–94. https://doi.org/10.1007/BF02249046

• Abstract: Current software cost estimation models, such as the 1981 Constructive Cost Model (COCOMO) for software cost estimation and its 1987 Ada COCOMO update, have been experiencing increasing difficulties in estimating the costs of software developed to new life cycle processes and capabilities. These include non-sequential and rapid-development process models; reuse-driven approaches involving commercial off-the-shelf (COTS) packages, re-engineering, applications composition, and applications generation capabilities; object-oriented approaches supported by distributed middleware; and software process maturity initiatives. This paper summarizes research in deriving a baseline COCOMO 2.0 model tailored to these new forms of software development, including rationale for the model decisions. The major new modeling capabilities of COCOMO 2.0 are a tailorable family of software sizing models, involving Object Points, Function Points, and Source Lines of Code; nonlinear models for software reuse and re-engineering; an exponentdriver approach for modeling relative software diseconomies of scale; and several additions, deletions and updates to previous COCOMO effort-multiplier cost drivers. This model is serving as a framework for an extensive current data collection and analysis effort to further refine and calibrate the model's estimation capabilities.

The Linux Foundation Releases First-Ever Value of Collaborative Development Report[edit | edit source]

The Linux Foundation Releases First-Ever Value of Collaborative Development Report. (2015, September 30). Linux Foundation. https://linuxfoundation.org/press-release/the-linux-foundation-releases-first-ever-value-of-collaborative-development-report/

==== Quantifying the Value of Open Source Hard-ware Development ====Pearce, J. M. (2015). Quantifying the Value of Open Source Hard-ware Development. Modern Economy, 6(1), 1–11. https://doi.org/10.4236/me.2015.61001

• Abstract: With the maturation of digital manufacturing technologies like 3-D printing, a new paradigm is emerging of distributed manufacturing in both scientific equipment and consumer goods. Hardware released under free licenses is known as free and open source hardware (FOSH). The availability of these FOSH designs has a large value to those with access to digital manufacturing methods and particularly for scientists with needs for highly-customized low-volume production products. It is challenging to use traditional funding models to support the necessary investment of resources in FOSH development because of the difficulty in quantifying the value of the result. In order to overcome that challenge and harvest the current opportunity in both low-cost scientific equipment and consumer products, this article evaluates the following methods to quantify the value of FOSH design including: 1) downloaded substitution valuation; 2) avoided reproduction valuation and 3) market savings valuation along with additional benefits related to market expansion, scientific innovation acceleration, educational enhancement and medical care improvement. The strengths and weaknesses of these methods are analyzed and the results show that the methods are relatively straight-forward to institute, based on reliable freely-available data, and that they minimize assumptions. A case study of a syringe pump with numerous scientific and medical applications is presented. The results found millions of dollars of economic value from a relatively simple scientific device being released under open-licenses representing orders of magnitude increase in value from conventional proprietary development. The inescapable conclusion of this study is that FOSH development should be funded by organizations interested in maximizing return on public investments particularly in technologies associated with science, medicine and education.

The Linux Kernel Open Source Project on Open Hub[edit | edit source]

The Linux Kernel Open Source Project on Open Hub : Estimated Cost Page. (n.d.). Retrieved February 28, 2021, from https://www.openhub.net/p/linux/estimated_cost

==== How nonprofits grow: A model ====James, E. (1983). How nonprofits grow: A model. Journal of Policy Analysis and Management, 2(3), 350-365.

• Abstract: When nonprofit organizations operate in a field of service that might otherwise be provided by government or by for‐profit organizations, such as schools, hospitals, or museums, their decisions on the type and amount of services to be delivered are motivated by factors that are quite distinctive. Because they must operate on a break‐even basis with revenues raised on a voluntary basis, nonprofit organizations typically take on activities from which they derive no satisfaction in order to subsidize activities that they regard as of higher value. As a result, the mix of services they offer and the charges they impose will generally be quite different from those of a government institution or a for‐profit organization.

Accounting for the value of volunteer contributions[edit | edit source]

Mook, L., Sousa, J., Elgie, S., & Quarter, J. (2005). Accounting for the value of volunteer contributions. Nonprofit Management and Leadership, 15(4), 401–415. https://doi.org/10.1002/nml.79

• Abstractː There is a need for an accounting paradigm that properly illustrates the value that nonprofits generate. Much of that value comes from volunteer contributions, which are significant but for the most part are not included in financial accounting statements, even though our research indicates that they account for almost a third of the value added by these organizations. This article reports the results of two studies related to measuring volunteer value in the accounting of nonprofits and then draws some policy implications from the research. The first study, a survey of 156 nonprofits in Canada, found that although about one‐third of the sample kept records of volunteer hours, only 3 percent included a value for them in their accounting statements. The second study, of nonprofit accountants, found that they did not feel that financial accounting statements properly represented the contribution of their organizations. A series of policy recommendations are presented, including suggestions for revising the regulations of accounting bodies for imputing volunteer value and creating accounting statements that better represent the contribution of nonprofits.

Engineers Without Borders USA[edit | edit source]

Mission & History. (n.d.). Engineers Without Borders USA. Retrieved March 3, 2021, from https://www.ewb-usa.org/mission-and-history/

==== Engineers Without Borders USA ====Financials. (n.d.). Engineers Without Borders USA. Retrieved March 3, 2021, from https://www.ewb-usa.org/financials/

==== Green Card for Immigrant Investors ====Green Card for Immigrant Investors | USCIS. (2020, June 16). https://www.uscis.gov/green-card/green-card-eligibility/green-card-for-immigrant-investors

==== Software Piracy: Estimation of Lost Sales and the Impact on Software Diffusion ====Givon, M., Mahajan, V., & Muller, E. (1995). Software Piracy: Estimation of Lost Sales and the Impact on Software Diffusion. Journal of Marketing, 59(1), 29–37. https://doi.org/10.1177/002224299505900103

• Software piracy by users has been identified as the worst problem facing the software industry today. Software piracy permits the shadow diffusion of a software parallel to its legal diffusion in the marketplace, increasing its user base over time. Because of this software shadow diffusion, a software firm loses potential profits, access to a significant proportion of the software user base, opportunities for cross-selling, and marketing its other products and new generations of the software. However, shadow diffusion may influence the legal diffusion of the software. Software pirates may influence potential software users to adopt the software, and some of these adopters may become buyers. A diffusion modeling approach is suggested to track shadow diffusion and the legal diffusion of a software over time. The approach enables management to estimate (1) the pirated adoptions over time and (2) the percentage of legal adoptions due to the influence of pirates. The modeling approach is applied to study the diffusion of two types of software (spreadsheets and word processors) in the United Kingdom. The results suggest that although six of every seven software users utilized pirated copies, these pirates were responsible for generating more than 80% of new software buyers, thereby significantly influencing the legal diffusion of the software. The implications of these results are discussed.

Average Price Paid for a Learning Management System[edit | edit source]

Menard, J. (2020). Average Price Paid for a Learning Management System. LISTedTECH. https://www.listedtech.com/blog/average-cost-of-an-lms

==== LISTedTECH: Higher EdSystems & EdTech Intelligence ====LISTedTECH. (n.d.). LISTedTECH: Higher EdSystems & EdTech Intelligence. LISTedTECH. Retrieved March 20, 2021, from https://www.listedtech.com

==== Mathematica Price for Colleges & Universities: Individual License Options ====WOLFRAM. (2021). Mathematica Price for Colleges & Universities: Individual License Options. https://www.wolfram.com/mathematica/pricing/colleges-universities/

==== NX Cloud Connected Products ====Siemens. (2020). NX Cloud Connected Products. http://web.archive.org/web/20210305204203/https://www.dex.siemens.com/plm/nx-cloud-connected-products?viewState=ListView&cartID=&portalUser=&store=&cclcl=en_US&selected=plm==== MoodleCloud ====Moodle. (2020). MoodleCloud. Moodle. https://moodle.com/moodlecloud/

==== Compare Office 365 Education Plans ====Microsoft. (2021). Compare Office 365 Education Plans. https://www.microsoft.com/en-us/microsoft-365/academic/compare-office-365-education-plans

==== HOUR town: Paul Glover and the genesis and evolution of Ithaca HOURS ====Jacob, J., Brinkerhoff, M., Jovic, E., & Wheatley, G. (2004). HOUR town: Paul Glover and the genesis and evolution of Ithaca HOURS. International Journal of Community Currency Research, 8, 29-41.

• Abstractː Ithaca HOURS are, arguably, the most successful of the local currency experiments of the last two decades. At the height of their popularity in the mid-1990s, perhaps as many as 2,000 of Ithaca and region's 100,000 residents were buying and selling with HOURS. The high profile of HOURS in the Ithaca community has prompted a series of articles, television news segments and documentaries, primarily for the popular media. Though constituting valuable documentation of an intrinsically interesting phenomenon, these reports has tended to be fragmentary and ahistorical, thus lacking in context in terms of the longitudinal evolution of the Ithaca region's political economy. The present study attempts to remedy these lacunae in our understanding of the genesis and evolution of Ithaca HOURS by presenting a systematic account of Ithaca's experiment with local currencies over the past decade and a half through the person of Paul Glover, the individual most closely associated with the founding and developing of HOURS. The article follows the activist career of Glover through the end of 2003, thus placing HOURS in the context of Ithaca's activist community's efforts to push the local polity and economy in the direction of ecological sustainability
• P3

The social and cultural capital of community currency: an Ithaca HOURS case study survey[edit | edit source]

Jacob, J., Brinkerhoff, M., Jovic, E., & Wheatley, G. (2004). The social and cultural capital of community currency: an Ithaca HOURS case study survey. International Journal of Community Currency Research, 8(4), 42-56.

• Abstractː In this article the authors report and analyze the data from an interview survey of 42 Ithaca HOURS community currency users. The theoretical context for the study is social capital, and the survey seeks to answer questions centering around the extent the interviewees participate in networks of reciprocity, trust and support. The survey results indicate that the respondents highly value their experiences buying and selling with HOURS, and that it does in fact function as a social capital resource for them. Nevertheless, on average, the respondents' use of HOURS was modest at best, with $300 to $350, exchanged in the 12 months prior to the survey. Since the exchange in HOURS is dwarfed by the mainstream economy's circulation of federal dollars, and since the respondents use of HOURS, on average, is only a very small part of their disposable income, the authors sought the significance of the HOURS economy in cultural and symbolic rather than material terms.

Fees for Visa Services[edit | edit source]

U.S. DEPARTMENT OF STATE. (n.d.). Fees for Visa Services. Retrieved March 21, 2021, from https://travel.state.gov/content/travel/en/us-visas/visa-information-resources/fees/fees-visa-services.html

==== Not all programmers are created equal ====Bryan, G. E. (1994). Not all programmers are created equal. Proceedings of 1994 IEEE Aerospace Applications Conference Proceedings, 55–62. https://doi.org/10.1109/AERO.1994.291203

• Abstractː Data measuring individual programmer performance was gathered over a 12-year period in a stable work group responsible for developing and supporting a single 4.2-million-line operating system and associated program products, communication systems, and databases. Analysis shows a wide variation in productivity from best- to poorest-performing individuals and groups. Eight percent of the work result was done by a single programmer out of a workforce of almost 200. A variation of 200:1 separated the top programmer from the poorest performers. The top 27% of programmers did 78% of the work. Planning for average productivity can therefore result in significant errors in projects.

Blockchain for program code credit and programmer contribution in a collective[edit | edit source]

Kozloski, J. R., Pickover, C. A., & Weldemariam, K. (2019). U.S. Patent No. 10,438,170. Washington, DC: U.S. Patent and Trademark Office.

• Abstractː A system for producing a computer program code collaboratively using blockchain includes a plurality of computer nodes, the plurality of computer nodes forming a distributed network for collaborative work. Each of the computer nodes communicates directly with the others, and is operated by a user in accordance with a common smart contract. Contributions of each of the users are entered into the blockchain at respective computer nodes as blocks when transactions have been completed in accordance with the following steps: writing code for inclusion in said computer software program; submitting the code for the computer software program to the distributed network to complete a transaction to add a block with the code to the blockchain of the computer software program; detecting by the distributed network of the submission of code for the computer software program; and adding the code as a block to the blockchain of the computer software program.

Measuring author contributions to the Wikipedia[edit | edit source]

Adler, B. T., de Alfaro, L., Pye, I., & Raman, V. (2008). Measuring author contributions to the Wikipedia. Proceedings of the 4th International Symposium on Wikis, 1–10. https://doi.org/10.1145/1822258.1822279

• Abstractː We consider the problem of measuring user contributions to versioned, collaborative bodies of information, such as wikis. Measuring the contributions of individual authors can be used to divide revenue, to recognize merit, to award status promotions, and to choose the order of authors when citing the content. In the context of the Wikipedia, previous works on author contribution estimation have focused on two criteria: the total text created, and the total number of edits performed. We show that neither of these criteria work well: both techniques are vulnerable to manipulation, and the total-text criterion fails to reward people who polish or re-arrange the content. We consider and compare various alternative criteria that take into account the quality of a contribution, in addition to the quantity, and we analyze how the criteria differ in the way they rank authors according to their contributions. As an outcome of this study, we propose to adopt total edit longevity as a measure of author contribution. Edit longevity is resistant to simple attacks, since edits are counted towards an author's contribution only if other authors accept the contribution. Edit longevity equally rewards people who create content, and people who rearrange or polish the content. Finally, edit longevity distinguishes the people who contribute little (who have contribution close to zero) from spammers or vandals, whose contribution quickly grows negative.

Code ownership in open-source software[edit | edit source]

Foucault, M., Falleri, J.-R., & Blanc, X. (2014). Code ownership in open-source software. Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, 1–9. https://doi.org/10.1145/2601248.2601283

• Abstractː Context: Ownership metrics measure how the workload of software modules is shared among their developers. They have been shown to be accurate indicators of software quality. Objective: Since ownership metrics studies were done only on industrial software projects, we replicated such a study on Java free/libre and open source software (FLOSS) projects. Our goal was to generalize an "ownership law" that stated that minor developers should be avoided. Method: We explored the relationship between ownership metrics and fault-proneness on seven FLOSS projects, using publicly available corpora to retrieve the fault-related information. Results: In our corpus, the relationship between ownership metrics and module faults is weak. At best, less than half of projects exhibit a significant correlation, and at worst, no projects at all. Moreover, fault-proneness seems to be much more influenced by module size than by ownership. Conclusion: The results of ownership studies done on closed-source projects do not generalize to FLOSS projects. To understand the reasons for that, we performed an in-depth analysis and found that the lack of correlation between ownership metrics and module faults is due to the distributions of contributions among developers and the presence of "heroes" in FLOSS projects.

Impact of Programming Features on Code Readability[edit | edit source]

Tashtoush, Y., Odat, Z., Alsmadi, I., & Yatim, M. (2013). Impact of Programming Features on Code Readability. Computer Science Faculty Publications, 7(6). https://doi.org/10.14257/ijseia.2013.7.6.38

• Readability is one important quality attributes for software source codes. Readability has also significant relation or impact with other quality attributes such as: reusability, maintainability, reliability, complexity, and portability metrics. This research develops a novel approach called Impact of Programming Features on Code Readability (IPFCR), to examine the influence of various programming features and the effect of these features on code readability. A code Readability Tool (CRT) is developed to evaluate the IPFCR readability features or attributes. In order to assess the level if impact that each one of the 25 proposed readability features may have, positively or negatively on the overall code readability, a survey was distributed to a random number of expert programmers. These experts evaluated the effect of each feature on code readability, based on their knowledge or experience. Expert programmers have evaluated readability features to be ordered then classified into positive and negative factors based on their impact on code readability or understanding. The survey responses were analyzed using SPSS statistical tool. Most of proposed code features showed to have significantly positive impact on enhancing readability including: meaningful names, consistency, and comments. On the other hand, fewer features such as arithmetic formulas, nested loops, and recursive functions showed to have a negative impact. Finally, few features showed to have neutral impact on readability.

Learning a Metric for Code Readability[edit | edit source]

Buse, R. P. L., & Weimer, W. R. (2010). Learning a Metric for Code Readability. IEEE Transactions on Software Engineering, 36(4), 546–558. https://doi.org/10.1109/TSE.2009.70

• Abstractː In this paper, we explore the concept of code readability and investigate its relation to software quality. With data collected from 120 human annotators, we derive associations between a simple set of local code features and human notions of readability. Using those features, we construct an automated readability measure and show that it can be 80 percent effective and better than a human, on average, at predicting readability judgments. Furthermore, we show that this metric correlates strongly with three measures of software quality: code changes, automated defect reports, and defect log messages. We measure these correlations on over 2.2 million lines of code, as well as longitudinally, over many releases of selected projects. Finally, we discuss the implications of this study on programming language design and engineering practice. For example, our data suggest that comments, in and of themselves, are less important than simple blank lines to local judgments of readability.

Direct and indirect knowledge spillovers: the "social network" of open-source projects[edit | edit source]

Fershtman, C., & Gandal, N. (2011). Direct and indirect knowledge spillovers: the "social network" of open-source projects. The RAND Journal of Economics, 42(1), 70–91. https://doi.org/https://doi.org/10.1111/j.1756-2171.2010.00126.x

• Abstractː Knowledge spillovers are a central part of knowledge accumulation. The article focuses on spillovers that occur through the interaction between different researchers or developers who collaborate on different research projects. The article distinguishes between project spillovers and contributors' spillovers and between direct and indirect spillovers. The article constructs a unique data set of open source software projects. The data identify the contributors who work on each project and thus enable us to construct a two‐mode network: a project network and a contributor network. The article demonstrates that the structure of these networks is associated with project success and that there is a positive association between project closeness centrality and project success. This suggests the existence of both direct and indirect project knowledge spillovers. We find no evidence for any association between contributor closeness centrality and project success, suggesting that contributor spillovers play a lesser role in project success.
• Spillover among teams (P2)
• Download measurement as an indication of success of a OSS (P3)

U.S. Patent No. 10,438,170[edit | edit source]

Kozloski, J. R., Pickover, C. A., & Weldemariam, K. (2019). U.S. Patent No. 10,438,170. Washington, DC: U.S. Patent and Trademark Office.

• Permissionless (P6)
• Considered to be unbreakable (P6)
• Avoid the need of trusted third party (P7)
• Use cryptocurrency system to handle FOSS collaboration (P7)
• No single entity has control over the entire system (P7)
• Trigger in user interface to detect completion-like action (P7)
• Other tools for monitoring contribution in a project does not adapt well with new implementations (P8)
• Tokens to identify the accessor (P9)
• Like Sensorica, considered the role of the programmer and their historical contribution (P9-10)

AutoCAD. (n.d.). AutoCAD Software| Get Prices & Buy Official AutoCAD 2022. Retrieved April 1, 2021, from[edit | edit source]

https://www.autodesk.com/products/autocad/overview

SketchUp. (n.d.). 3D Modeling Software Pricing| 3D Design Program Cost. SketchUp. Retrieved April 1, 2021, from[edit | edit source]

https://www.sketchup.com/plans-and-pricing

NX Cloud Connected Products. (n.d.). Retrieved April 1, 2021, from[edit | edit source]

https://www.dex.siemens.com/plm/nx-cloud-connected-products?viewState=ListView&cartID=&portalUser=&store=&cclcl=en_US&selected=plm

Polaris Office. (n.d.). Purchase and compare products - Polaris Office. Retrieved April 1, 2021, from[edit | edit source]

https://www.polarisoffice.com/en/store

Zoho. (n.d.). Online file Storage - Document Management Software| Zoho Docs. Zoho. Retrieved April 3, 2021, from[edit | edit source]

https://www.zoho.com/docs/zoho-docs-pricing.html

Microsoft. (n.d.). Compare Microsoft 365 Enterprise plans. Retrieved April 1, 2021, from[edit | edit source]

https://www.microsoft.com/en-us/microsoft-365/compare-microsoft-365-enterprise-plans

Walmart. (n.d.). Purchase Microsoft Office. Walmart.Com. Retrieved April 3, 2021, from[edit | edit source]

https://www.walmart.com/search/search-ng.do?search_query=Purchase%20Microsoft%20Office&&adid=22222222224210561490&wmlspartner=wmtlabs&wl0=e&wl1=o&wl2=c&wl3=75247860408290&wl4=kwd-15082655495:aud-807615483:loc-190&wl5=110887&wl6=&wl7=&wl14=purchase%20microsoft%20office&veh=sem&msclkid=5f4ebdc713c41f8ed1feddafde3b8e96&gclid=5f4ebdc713c41f8ed1feddafde3b8e96&gclsrc=3p.ds

SourceForge - Download, Develop and Publish Free Open Source Software. (n.d.). Retrieved April 1, 2021, from[edit | edit source]

https://sourceforge.net/

Apache Open Office. (n.d.). Why Apache OpenOffice. Retrieved April 1, 2021, from[edit | edit source]

https://www.openoffice.org/why/index.html

SourceForge. (n.d.). Apache OpenOffice Download Statistics by OS: All Files. Retrieved April 1, 2021, from[edit | edit source]

https://sourceforge.net/projects/openofficeorg.mirror/files/stats/os?dates=2011-11-30+to+2021-04-01

GitHub. (n.d.). apache/openoffice. GitHub. Retrieved April 1, 2021, from[edit | edit source]

https://github.com/apache/openoffice

FreeCAD. (n.d.). FreeCAD: Your own 3D parametric modeler. Retrieved April 1, 2021, from[edit | edit source]

https://www.freecadweb.org/

TurboCAD. (n.d.). TurboCAD.com - Optimize Design Workflow - TurboCAD via IMSI Design. Retrieved April 5, 2021, from[edit | edit source]

https://www.turbocad.com/

CorelCAD. (n.d.). CorelCAD 2021| Get Your Free Trial. Retrieved April 5, 2021, from[edit | edit source]

https://www.coreldraw.com/en/product/corel-cad/

DarrenP. (2021, March 8). Re: Perpetual License?[edit | edit source]

https://forums.autodesk.com/t5/installation-licensing/perpetual-license/m-p/10139894#M251757

SENSORICA[edit | edit source]

Peer Into The Future[edit | edit source]

peer into the future - Q and A. (n.d.). Retrieved March 8, 2021, from https://www.sensorica.co/help/qanda

==== Recognizing the Magic of Nonprofit Volunteers ====K. J. Bulman. (2018, May 29). Recognizing the Magic of Nonprofit Volunteers. NonProfit PRO. https://www.nonprofitpro.com/post/recognizing-the-magic-of-nonprofit-volunteers-and-those-who-lead-them/

==== National Council of Nonprofits ====Volunteers. (2015, January 6). National Council of Nonprofits. https://www.councilofnonprofits.org/tools-resources/volunteers

==== Immigrant Investor Visas ====Immigrant Investor Visas. (n.d.). Retrieved March 12, 2021, from https://travel.state.gov/content/travel/en/us-visas/immigrate/immigrant-investor-visas.html

==== EB-5 Immigrant Investor Program ====EB-5 Immigrant Investor Program | USCIS. (2020, June 10). https://www.uscis.gov/working-in-the-united-states/permanent-workers/eb-5-immigrant-investor-program

==== Chapter 23 - Translating Commons-Based Peer Production Values into Metrics: Toward Commons-Based Cryptocurrencies ====De Filippi, P. (2015). Chapter 23 - Translating Commons-Based Peer Production Values into Metrics: Toward Commons-Based Cryptocurrencies. In D. Lee Kuo Chuen (Ed.), Handbook of Digital Currency (pp. 463–483). Academic Press. https://doi.org/10.1016/B978-0-12-802117-0.00023-0

• Abstractː Commons-based peer production (CBPP) constitutes today an important driver for innovation and social and cultural development, both online and off-line, through the establishment of an alternative, commons-based ecosystem, relying on peer production and collaboration among peers contributing toward a common good. Yet, to the extent that it operates outside of the market economy, the value of CBPP cannot be understood by relying exclusively on traditional market mechanisms (such as pricing). Based on empirical research on emerging value forms in the context of CBPP, we seek to achieve a better understanding of the value produced by CBPP communities, so as to come up with an alternative, universal, denominator of value that could act as an interface between the commons-based ecosystem and the market economy.

ROI[edit | edit source]

Return on investment for open source scientific hardware development[edit | edit source]

Pearce, J. M. (2016). Return on investment for open source scientific hardware development. Science and Public Policy, 43(2), 192–195. https://doi.org/10.1093/scipol/scv034

• Abstractː The availability of free and open source hardware designs that can be replicated with low-cost 3D printers provides large values to scientists who need highly-customized low-volume production scientific equipment. Digital manufacturing technologies have only recently become widespread and the return on investment (ROI) was not clear, so funding for open hardware development was historically sparse. This paper clarifies a method for determining an ROI for the development of scientific free and open source hardware (FOSH). By using an open source hardware design that can be manufactured digitally, the relatively minor development costs result in enormous ROIs for the scientific community. A case study is presented of a syringe pump released under open license, which results in ROIs for funders ranging from hundreds to thousands of percent after only a few months. It is clear that policies encouraging FOSH scientific hardware development should be adopted by organizations interested in maximizing return on public investments for science.
• Funding FOSH induces a large ROI

Chapter 24 - Measuring the Returns to R&D[edit | edit source]

Hall, B. H., Mairesse, J., & Mohnen, P. (2010). Chapter 24 - Measuring the Returns to R&D. In B. H. Hall & N. Rosenberg (Eds.), Handbook of the Economics of Innovation (Vol. 2, pp. 1033–1082). North-Holland. https://doi.org/10.1016/S0169-7218(10)02008-3

• Abstractː We review the econometric literature on measuring the returns to R&D. The theoretical frameworks that have been used are outlined, followed by an extensive discussion of measurement and econometric issues that arise when estimating the models. We then provide a series of tables summarizing the major results that have been obtained and conclude with a presentation of R&D spillover returns measurement. In general, the private returns to R&D are strongly positive and somewhat higher than those for ordinary capital, while the social returns are even higher, although variable and imprecisely measured in many cases.
• R&D develop new applications and innovations (P4)
• R&D positive spillovers (P4)
• Non-pecuniary spillovers from R&D's knowledge development (P5)
• Return to R&D is variant (P5)
• Failed R&D (P28)
• Returns and social returns

A New Look at the Returns and Risks to Pharmaceutical R&D[edit | edit source]

Grabowski, H., & Vernon, J. (1990). A New Look at the Returns and Risks to Pharmaceutical R&D. Management Science, 36(7), 804–821. https://doi.org/10.1287/mnsc.36.7.804

• Abstractː This study investigates the returns to R&D for 100 new drugs introduced into the United States during the decade of the 1970s. In contrast to prior studies, it incorporates several significant structural changes that have occurred in the pharmaceutical industry during the 1980s. These include higher real drug prices and a greater degree of generic competition. A major finding is that the return on R&D for the average new drug is approximately equal to the 9 percent industry cost of capital. However, the performance of new drugs introduced during the latter half of the 1970s was markedly better than that of early 1970s introductions. This latter finding is consistent with the more rapid rate of industry growth in real R&D expenditures. The study also finds that the variation in returns is highly skewed, with only the top 30 drugs covering mean R&D costs on a fully allocated basis. Finally, it is shown that real drug price increases in the 1980s were necessary for the average new drug introduction to recover its R&D costs.

Return on Investment[edit | edit source]

Phillips, P. P., & Phillips, J. J. (2009). Return on Investment. In Handbook of Improving Performance in the Workplace: Volumes 1-3 (pp. 823–846). John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470592663.ch53

• Abstractː The need for data that satisfy the taste of various stakeholders has positioned the return on investment (ROI) methodology as a leading technique to measure human performance technology (HPT) success. The methodology is used to evaluate program success in private‐sector, public‐sector, and non‐profit organizations. While ROI is the ultimate measure developed through the ROI methodology, it is only one of six measures. This chapter introduces the ROI methodology and describes how it is can be used to report HPT success. The most fundamental measure used to allocate resources is either the benefit‐cost ratio (BCR) or ROI percentage. These metrics have been used for centuries to show the ultimate profitability of a program, process, or initiative. The ROI process follows ten steps that occur in four phases: evaluation planning, data collection, data analysis, and reporting.
• Benefit-cost ratio (P9)
• Socially educating ROI methodology to make it more acceptable (P21)

Return of Investment (ROI) in Research and Development (R&D): Towards a framework[edit | edit source]

Hassanzadeh, M., & Bigdeli, T. B. (2019). Return of Investment (ROI) in Research and Development (R&D): Towards a framework. Collaboration – Impact on Productivity and Innovation: Proceedings of 14th International Conference on Webometrics, Informetrics and Scientometrics & 19th COLLNET Meeting 2018, December 5-8, 2018, University of Macau, Macau, 31–39. https://doi.org/10.22032/dbt.39330

• Abstractː There is a significant relationship between the development of countries and their investment on research and development (R&D). Often, investments on science is seen as a resource of national prestige in countries. An outstanding part of spending go to the universities as part of science policy utilization. For that, most of investigations on return of investments (ROI) on science focus on universities as the main authorities of scientific endeavor. The main issue is that, calculating ROI of R&D is not as simple as the industry of service sectors. In industry and service input and output of the process is clearly calculable. For example a manger in industry sector is able to calculate the cost of product and services. But in science it is not simple to calculate the cost of products because of variety and complexity of resources allocated for its production. A paper as an output of a scientific endeavor easily is not calculable in terms of financial formula. Sometimes a scientific finding save the life of millions of people and provide society with bunch of opportunities and guide the communities towards prosperous life. These are not simply calculable in term of ROI investigations. Most of scientometrics studies focuses on the determined document based outputs of the R&D systems. Since financial inputs play crucial role in progress of R&D systems, taking them in account when calculating the efficacy of this system will provide us with clearer image of R&D performance. This research aimed to focus more on financial aspects of R&D performance in universities forward utilizing some sort of knowledge economy framework. For that, we strive to formulate the expenditure has been carried out in various levels of a university which results in a unit of scientific paper, highly cited paper, fruitful actors, and etc. we have focused on the Tarbiat Modares University (TMU) a fully accredited state university for graduate studies with more than 7000 students and almost 1000 faculty members. TMU always ranks between top 3 Iranian universities.
• Social return (P2)

The relationship between performance indicators for academic research and funding: Developing a measure of return on investment in science[edit | edit source]

Lindsey, D. (1991). The relationship between performance indicators for academic research and funding: Developing a measure of return on investment in science. Scientometrics, 20(1), 221–234. https://doi.org/10.1007/BF02018156

• Abstractː Public universities reflect the aspirations a state or society has for its young people and for itself. In this study our interest has been to examine the level of public funding for universities and its relation to quality. In order to do this we collected funding data for a sample American universities. Additionally, we collected data on the production of science by faculty at the institutions in our American sample. The results indicated a strong relation between investment in higher education and quality. We then developed a measure of return on investment in research which combined these measures of funding and research production. We conclude by examining the nature of the relationship between funding and research quality at public universities.
• Measure universities' performances rather than ranking for funding (P1)
• Low production due to low funding for the University of Oregon (P11)
• UoO has a high ROI from their method (P11)

Changes in Quality among Public Universities[edit | edit source]

Volkwein, J. F. (1989). Changes in Quality among Public Universities. The Journal of Higher Education, 60(2), 136–151. https://doi.org/10.2307/1982174

• Abstractː This analysis extends earlier research by examining three questions about the relationships between campus autonomy and measures of faculty quality, student quality, and external funding. The results suggest that changes in these measures of university quality are more influenced by state financial support than by state regulation.
• The less manage the better (P4)
• Regulation leads to greater cost (P4)
• Stage regulation does not improve quality for universities (P10)
• Size of universities have more to be with their effectiveness and quality (P11)

The rate of return to investment in R&D: The case of research infrastructures[edit | edit source]

Del Bo, C. F. (2016). The rate of return to investment in R&D: The case of research infrastructures. Technological Forecasting and Social Change, 112, 26–37. https://doi.org/10.1016/j.techfore.2016.02.018

• Abstractː The return to R&D investment and activities has been the object of a vast literature, both from a theoretical and empirical perspective. The aim of this overview is to present a selection of contributions to underscore the main shared findings and highlight open issues, while also providing a preliminary analysis of the returns to R&D investment in large research infrastructures (RIs) in Europe. First, a common methodological framework is distilled from the macro-literature, examining the return to R&D in aggregate terms. Then, the evaluation in the context of specific projects, mainly in large RIs, is examined, followed by the explicit consideration of externalities and spillover effects of research activities. A novel empirical analysis of European RIs is also presented, based on a novel data set, to highlight trends and suggest new avenues for the evaluation of the rate of return to investments in research infrastructures, using both a cost effectiveness ratio and a bibliometric citation count as metrics to evaluate the return to R&D investment in these facilities. Directions for future research are sketched in the concluding section.