[go: nahoru, domu]

Jump to content

Wikipedia talk:Requests for arbitration/Pseudoscience/Workshop

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

Collaborative effort to propose principles, findings of facts, and remedies

[edit]

May I in typical Wiki openess of process call the alleged suppression-of-non-mainstream-science faction to conspire for some good formulations here?

I mean, we may reach 1MB of evidence soon, individually adding somes tens of proposals may be counterproductive.

I assume that BLP, NPA and related issues can be handled rather straightforwardly. We should show the utmost patience even with the contributors holding rather strange ideas.

For me the core issue is about how to interpret NPOV, "undue weight" and "reliable source".

My very first draft re principles would be:

  1. Wikipedia is an encyclopedia. As an encyclopedia it reports the established knowledge of its time.
  2. In the fields of mathematics and the physical science, established knowledge is what is taught at universities and included in contemporary textbooks

These two points should be complemented by a Wikipedia is not paper clause, ceding that also ideas and theories ignored or thought to be false by mainstream science, are valid subjects for Wikipedia, within their own articles article only (except for obsolote theories of historical signifance, which are valid to be mentioned in the main articles).

Pjacobi 22:29, 15 October 2006 (UTC)[reply]

You need to be careful about rewording principles/findings of fact/remedies after they have already attracted comment, especially if the wording changes the meaning in any way so that an earlier commenter might feel that the changed wording no longer reflects the intent at the time they commented. You might want to work on principles here or a subpage to avoid making too many changes once you post them "live" (so to speak). Thatcher131 22:49, 15 October 2006 (UTC)[reply]
For just that reason I didn't post them "live" (i.e. on the project page) but inivited discussion here first. --Pjacobi 08:25, 16 October 2006 (UTC)[reply]


  • This is a step in the right direction. But I think the solution is more complicated. Everyone agress with NPOV, Undue weight and Reliable sources, in principle.
  • We have to be careful describing certain knowledge as fact. This may be a SUBJECTIVE judgement, especially where scientific theories are concerned. For example, Encyclopædia Britannica, tells us that the "big-bang model" is a "widely held theory"[1] whereas the "Steady State theory" is "a view"[2]. Similarly, Encarta tells us that "Big Bang Theory" is the "currently accepted explanation"[3] where as the "Steady State theory" is "theory of cosmology .. once a rival to the big bang theory".[4] Note the lack of judgement.
  • I also note that good science found that the Big Bang is better than the Steady State theory. Some scientists still favour the Steady State theory, and although they are in the minority, it does not make it pseudoscience, nor bad science (and as far as I know, there are NO reliable sources suggesting otherwise).
  • Cosmologist Stephen Hawking wrote that Cosmology was once considered pseudoscience,[5]. This reliable source is factual, which does not imply that cosmology IS pseudoscience. How and where do we describe this fact?
  • Scientists are not familiar with many minority theories. How do we know whether other scientists have considered them and ignored them, or whether they are just not aware of them? Can we infer why?
  • How many scientists' views represent the "consenus" or "mainstream science"? Does it matter whether these views are peer reviewed?
  • Scientists have described the Wolf effect as a "redshift mechanism" (see article), there are only three other known redshift mechanisms. This is well-known in optics, not so well-known in astronomy. Is one view more important than another? --Iantresman 11:40, 16 October 2006 (UTC)[reply]
But I think the solution is more complicated. Everyone agress with NPOV, Undue weight and Reliable sources, in principle - I assume this is one of the point, we indeed agree on. The difficult part is how to interpret NPOV, Undue weight, Reliable sources (and enyclopedia) in practise. IMHO the past approach was just don't generate policy overload let the editor battle it out in concrete instance. True to the motto, that Wikipedia doesn't work in theory, but only in practise. This unfortunately generated some very messy cases: The Aetherometry incident, were outright deletion was considered to be only escape. The Bogdanov affair incident, were an atypical rule of revert newbies on sight had to be implemented. And a lot of unresolved problems, like how to NPOV Heim theory, and last but not least contributor exhaustion and retirement.
From your examples I would only pick up three:
  • Cosmology as pseudoscience: yes, if include the the view, that it belongs to philosophy (and theology, for theists), it indeed was once the wide consensus. Name calling Hawking as witness I'd consider bad style, but otherwise I'd agree that it belongs into history section -- isn't it included now? <trivia>Hamburg Public Libraries still use a topical classification, most likely originating from the 50s, where cosmology and astrology are grouped together.</trivia>
  • Big Bang vs Steady State: I don't see the judgement of the other encyclopedia less harsh than ours. once was is a rather clear statement.
  • Scientists are not familiar with many minority theories. How do we know whether other scientists have considered them and ignored them, or whether they are just not aware of them? Can we infer why? It's not our job to rise scientists' awareness of minority theories or speculate why they don't publish about them. The minority theory proponents have to advertise their theories to scientists themselves, we will only report on the results.
Pjacobi 12:10, 16 October 2006 (UTC)[reply]
Such results are firstly reported in peer reviewed journals, and only if a majority of scientists favour them are they likely to be included in textbooks. Consequently your proposal boils down to imposing a dated majority POV on Wikipedia, which is certainly against policy. It would in practice reduce Wikipedia to the level of textbooks. Then, if a student wants to know about other POV's, Wikipedia would be hiding the additional information that he/she is looking for. Not very useful and certainly against the purpose of Wikipedia.
Thus I'd say: established knowledge is what is taught at universities and included in contemporary textbooks as well as in science journals. Harald88 22:52, 19 October 2006 (UTC)[reply]
  • So we can accurate described Cosmology as being once considered pseudoscience, even though it is considered a well-accepted theory/model now. Reliable sources support this description. This statement is NPOV.
  • We can accurately described astrology as being considered pseudoscience by many people, again there are many reliable sources. This statement is NPOV.
  • Plasma cosmology, Tired light, Eric Lerner have all been described as pseudoscience in articles. In evidence, Intrinsic redshift was described as "Pie-in-the-sky fantasy", Halton Arp as a "pathological skeptic", Redshift quantization as due to "researchers [who] think may hold the key to getting rid of the Big Bang." None of these descriptions is based on any source, reliable or otherwise. Do we consider them to be NPOV?
  • I contend that many of the other descriptions of some minority scientific theories are also mis-represented, and not supported by reliable sources. And this is what I believe needs to be addressed. --Iantresman 15:39, 16 October 2006 (UTC)[reply]
The problem here is, how to (and whether) to transform negative statements into positive statements. Not being included in contemporary textbooks and not being currently taught at universities seems rather uncontroversial either. But negations are hard to source.
Also note, the worse the science, the more unlikely it is, that anybody will provide us with critical assessment. That lead to the Aetherometry deletion. But other problems are open, e.g. Hydrino theory. This case has the additional problem, that there is a definitive commercial interest in the content of the Wikipedia article.
16:05, 16 October 2006 (UTC)
  • Simple, you don't transform statements. To say that a theory is based on one peer reviewed paper, is a neutral fact. We don't say "only one", nor pretend it has been "well received", nor judge the paper either way, when verification is not available. Science is agnostic.
  • Suggesting a paper is "ignored by the scientific community", is "not included in contemporary text books", or "not taught at universities", are statements I would think are impossible to verify and subject to constant change. These statements are also negatively suggestive, which may not be the case.
  • I am sure there are many obscure peer reviweed papers, with few citations, that are also "ignored by the scientific community", are "not included in contemporary text books", are are "not taught at universities", not because they are bad, but because they are obscure, or science has not gotten around to looking at. --Iantresman 16:50, 16 October 2006 (UTC)[reply]
The textbook, and taught at universities criterion is still the best we have. Only for very ground breaking, new results, there may be a problem. It is better than weighting the relative merits peer reviewed papers, looking up citatation counts , or doing polls How many scientists' views represent the "consenus" or "mainstream science"? Does it matter whether these views are peer reviewed
Those who wish to include a topic will be bound to provide evidence for textbook coverage and taught at universities if doubts. Topics passing this test are first class citizens within our encyclopedia.
Pjacobi 17:23, 16 October 2006 (UTC)[reply]
I think that textbooks, university taught, peer review, university researched, are all important in supporting an article; by referencing all/any of these gives the reader the chance to drawn their own opnions on a subject, rather than rely on editors. --Iantresman 17:48, 16 October 2006 (UTC)[reply]

Proposed guidelines for statements of fact and different grades of facts/viewpoints

[edit]

(Was orignially a continuation of previous thread. See towards the end for SerpentsChoice's excellent proposal and discussion.)

Let me suggest a couple of guidelines:
  • nothing in a scientific article should be presented as fact, except sourced statements about who says what - even when it seems like a completely obvious fact.
bad: "humans evolved from apes"
good: "the prevalent theory held by most paleonthologists is that humans evolved from apes" (cite from textbook #1) (cite from textbook #2)
bad: "the speed of light is 299,792km/s":
good: "the speed of light in vacuum is believed to be a physical constant (cite #1). the current value is measured as 299,792km/s by X (cite #2)"
  • articles about a currently non-mainstream theory should devote at least half of the article to what proponents of that theory claim, and no more than half to criticism from the mainstream. the theory should be clearly labelled as non-mainstream, if that can be sourced (which will usually not be very easy, so please use limited-scope wording there: bad: "cold fusion is universally considered to be pseudoscience" good: "john huizenga, the head of the doe review panel, describe cold fusion as a pseudoscience" (cite)).
  • avoid all value adjectives such as mainstream (or non-mainstream), controversial, reputable, prominent, pseudoscientific, skeptic, supporter, ... outside of direct quotes. also avoid all universal claims: "there is no evidence..." "most scientists..." etc. these are usually the things that spark POV disputes. stick to a simple "X said Y" or "published paper A by B says that C" format.
ObsidianOrder 21:25, 16 October 2006 (UTC)[reply]
In principle, I think that sounds very good. I think it would be helpful to includes lots more examples where "value adjectives" are used, and how they can be replaced. And the guideline on non-mainstream proportionality I think should also suggest that we describe the theory first, and have critical sections following. --Iantresman 21:52, 16 October 2006 (UTC)[reply]
Without too much time to go into details, I have to state that I violently oppose the first suggestion of User:Iantresman. Linking "theory" to theory#Science (or scientific process#Evaluation and iteration) may be an option for the curious reader but (a) doing a Popper, Falsifiability, and Fallability in nutshell section in every article about a scientific theory is just crazy and (b) citing some expert witnesses with quotes gives just the wrong impression -- it's not the case the A, B and C and some ten others hold this view, but you can enroll at thousands of universities all over the world and you will be taugh this theory. --Pjacobi 21:00, 17 October 2006 (UTC)[reply]
I think if ObsidianOrder's suggestions were taken seriously, it would make Wikipedia much harder to read and use and misrepresent many issues. If an article on Wikipedia said that ObsidianOrder says that Wikipedia articles should be NPOV, it would misrepresent how NPOV is treated on Wikipedia. The speed of light is measured regularly and constantly, and is depended upon in a thousand ways in modern technology. To say "was measured by X" is to misrepresent it as one person's concept, not the vastly measured and tested concept that it is.
And there's no reason to limit this to science. The existance of the Holocaust is hotly debated; we could say that "Jews were killed at Auschwitz, according to Elie Wiesel." Our article on Elvis treats his death as established fact, as does our article on Julius Caesar. Shouldn't we attribute the accusation of their deaths to someone? In both those cases, it would be impossible to reestablish the alleged fact of their deaths, unlike the speed of light, which is rechecked frequently.--Prosfilaes 17:14, 19 October 2006 (UTC)[reply]
I assure you my suggestion was serious. As for its use outside of science topics: hadn't thought about that, but now that you mention it, might be a good idea. By the way, there is nothing wrong with describing a source in a way that makes its relative weight clear (for example: "for a survey of 50 recent highly accurate measurements of the speed of light, see X"). It seems my notion of what is a fact is a whole lot more strict than yours. If just one scientist reports some result, I assume you wouldn't claim it is a fact? But if some magical number of scientists greater than one report it, it becomes a fact? What is that magic number? ObsidianOrder 19:03, 23 October 2006 (UTC)[reply]
I picked the speed of light for a reason, it is the closest thing to a science fact, and yet even that is not universally accepted (see for example Webb et al, Phys. Rev. Lett., vol 87, p091301 (2001); Davies et al, Nature, vol 418, p602 (2002); Lamoreaux et al, Phys. Rev. D, vol 69, p121701 (2004)). Would it be NPOV for us to say it is constant? I hardly think so. ObsidianOrder 19:31, 23 October 2006 (UTC)[reply]
I would posit that implying mainstream thought as potentially non-factual -- whether in science or elsewhere -- in the absense of strongly compelling arguments to the contrary does Wikipedia a disservice in its role as an encyclopedia. It is certainly our place to comment on appropriately verifiable controversy and counter-arguments, but rendering every fact in the encyclopedia as nothing more than the cited opinion of one or more arbitrarily selected authorities is not, in fact, a neutral point of view. It is certainly possible (in fact, quite likely guaranteed) that some establsihed facts will be, in time, proven false, and we will have to make due correction. So it has been for every encyclopedia since their inception. Wikipedia can survive being proven wrong on occasion as ideas and ideals change. Its utility is endangered, however, if we write off the ability to commit to anything in the name of false neutrality. Serpent's Choice 05:54, 24 October 2006 (UTC)[reply]
SC: implying mainstream thought as potentially non-factual is NPOV, it is the only way to be NPOV, because there is pretty much always some level of contrary arguments, and usually a lot more than you would guess. "absense of strongly compelling arguments" - indeed, but please define compelling. Better: "in the absence of siginificant arguments" - where siginificant could be judged simply by the number of adherents within the field (in the case of non-constant speed of light, I can come up with 20-50 experimental and several hundred theoretical papers - which probably merits a very brief mention in the speed of light article, and an article of its own). Finally, a historical perspective: we see that mainstream thought is rarely factual ;) "rendering every fact in the encyclopedia as nothing more than the cited opinion of one or more arbitrarily selected authorities" - if it is not that, what the hell is it? not necessarily arbitrarily selected, and we do try to impose some structure and organization, but yes, that about sums it up. "the ability to commit to anything" - ok, please tell me what criteria do you propose for stating a "science fact"? I.e. a plain statement of the form "X happens" in a science topic? ObsidianOrder 16:06, 24 October 2006 (UTC)[reply]

If I understand OO's opinion, this would be OO's proposal for rewriting George Washington:

"George Washington (according to Jones, born: February 22, 1732[1]– according to Frederick, died: December 14, 1799[2]) is believed by many historians and biographers (Jones, Clark, Frederick, etc.)[3] to have led the self-described[4] American Continental Army[5] to a claimed victory[6] over what is reported by many historians to have been Britain[7] in the so-called American Revolutionary War [8] (described by most historians to have occured from 1775–1783[9]."

He's advocating for weasling to get around arguments regarding verifiability. It's an unacceptable game. --ScienceApologist 16:27, 24 October 2006 (UTC)[reply]

Close, not quite:
"George Washington (born: February 22, 1732[10]– died: December 14, 1799[11]) led[12] the American Continental Army[13] to a victory[14] over Britain in the American Revolutionary War [15] (1775–1783[16])."
I would say this is a good article. The reason why we can state these as flat assertions is because, well, is there significant disagreement? If yes, it may be noted (either right there or a bit lower down), for example: "while most historians consider it a victory, Smith[17] argues it is a draw because...". Is there really a parallel here to the speed-of-light situation (which is: probably a few hundred articles arguing non-constant, a few thousand arguing constant? i'm guessing at the numbers, i have not sampled the literature systematically) ObsidianOrder 17:03, 24 October 2006 (UTC)[reply]
Why do you think there's significant argument about the speed-of-light and not about that sentence? Even if the ratios are as you suggest, it's not an useful comparison; no journal is going to publish an article claiming the speed of light is constant unless they percieve a controversy, which they may well not. As for that sentence, the American Army didn't beat Britian; the French and Americans did. I'm sure there's arguments that the Revolutionary War didn't start in 1775 at least at the same level, and people who will claim that using American to refer to the US is offensive. I'm sure some of the Mason conspiracists suggest that GW lived but hid himself to more effectively run the Illumanti in the US.--Prosfilaes 12:50, 25 October 2006 (UTC)[reply]

The problem with this polyanna vision of how to edit is that trolls can easily come in and disrupt Wikipedia by claiming there is a controversy about every plainly-stated fact in the encyclopedia. This is, frankly, unacceptable. No encyclopedia is written to avoid all controversy. It plainly cannot be done. There are bound to be wackos, loons, and whackjobs committed to finding controversy about whatever statements you wish to make. So instead of bending over backwards trying to accomodate all points of view, Wikipedia should be dedicated to marginalizing the marginal.--ScienceApologist 17:29, 24 October 2006 (UTC)[reply]

Controversies are surely verifiable?[6] Otherwise it's opinion? --Iantresman 17:49, 24 October 2006 (UTC)[reply]
Verifying the existence of controversies is fine for referencing such facts in articles, but we're talking about style rather than substance. --ScienceApologist 18:12, 24 October 2006 (UTC)[reply]
One thing that I think all of us should keep in mind as we work to look past our own individual views and biases: Wikipedia is not earning respect as an encyclopedia. Encyclopedias concede to a certain level of the presumption of fact. The earth is an oblate spheroid. Elvis Presley died on August 16, 1977. Project Apollo included the first moon landing on July 20, 1969. It is fine for an encyclopedia to talk about the people who believe otherwise -- who think the world is flat, who think Elvis is still alive (or died in the 90s), who think the Apollo landing was a Hollywood mockup. There is a difference between dissent and controversy. The shape of the earth, the death of Elvis, the moon landing -- these things are not topics of controversy, though they are topics of dissent. We do a disservice to our readers when we present dissent as controversy or conversely, when we express as controversy what is established to be true, or even what is established to be overwhelmingly likely. Controversy -- real, documented, legitimate controversy -- is as important as what is currently believed; to say that dissent does not deserve equal seating at the table is not a failure of NPOV. It is, rather, an underscoring of what this project hopes to achieve. Serpent's Choice 05:14, 25 October 2006 (UTC)[reply]
SC: Let me say first that I think you are doing awesome work at proposing/clarifying/formulating policy. That's why I will continue to press you ;) Yes, I am acting as a devil's advocate here, but not just to prove a point.
"Encyclopedias concede to a certain level of the presumption of fact. The earth is an oblate spheroid. Elvis Presley died on August 16, 1977. Project Apollo included the first moon landing on July 20, 1969." - ok. What criteria does something need to meet in order to be entitled to the presumption of fact? At least so far as the wording of the article is concerned? I am primarily interested in the answer for science-related articles, although the general answer is also interesting.
"It is fine for an encyclopedia to talk about the people who believe otherwise ... There is a difference between dissent and controversy." - how do you tell the two apart? I am interested primarily in an operational definition, even if only as a rule-of-thumb, as it will reduce strife.
Just to pick a couple of examples that I am interested in how you'd handle: creationism - a widely held opinion among non-experts [7], vastly less popular (dare I say absent?) among biologists. AIDS reappraisal - see this list of proponents and sympathizers. cold fusion - probably the example I'm most familiar with. ObsidianOrder 07:43, 25 October 2006 (UTC)[reply]

(Dropping indent to make this legible) I'm glad to see a devil's advocate. I think that, whatever is determined about this actual case by ArbCom, there will be discussions raised regarding the conclusions on the policy pages. Policy is based on consensus, after all. And clearly, there has to be some sort of real guidance here. A "know it when I see it" test won't do at all. We are bound in our actions, as well; many of the tools that can be used elsewhere would be original research in this context or at the least would presuppose a point of view. But, in the end, I still think this is possible.

I posit a series of loose standings that roughly parallel my thoughts on the different degrees of article treatment. Feel free to skewer. I know this could use some work.

  1. Notability - A viewpoint is notable if it meets whatever minimum requirements for inclusion deemed necessary by project consensus. Topics which are exclusively self-published or which have only a single identifiable adherent are unlikely to be considered notable and cannot have further standing regardless.
  2. Recognition - A notable viewpoint is recognized if one of the following is true: a) it has no notable counter-viewpoint OR b) is described as accepted, established, mainstream, or wholly factual in multiple qualified sources(1) within the topic's primary community OR c) is covered in multiple non-trivial sources equivalent(2) to those of its strongest counter-viewpoint. It is possible for a viewpoint to be recognized in some contexts but not in others (such as an unusual application of an otherwise mainstream theory or practice). When there is still substantial controversy or debate over a topic, there may be two or more recognized viewpoints without an established one.
  3. Establishment - A recognized viewpoint is established if it has received non-trivial treatment as accepted, established, mainstream, or wholly factual in multiple qualified sources both within and beyond the topic's primary community AND has no recognized counter-viewpoints that are also so presented. No two topics or statement in direct opposition to each other can be simultaneously established.

1Qualified sources for standing determination should demonstrate some effort to identify topics as mainstream or factual. Textbooks, government policies, and respected tertiary sources are examples of sources that are typically qualified. Self-published sources or those with demonstrable extremist bias are not qualified sources; popular media is generally qualified only when the topic is related to the media. 2Equivalent sources cover similar or overlapping topics and have similar renown or regard. In many science communities, equivalent sources have reasonably similar impact ratings. Popular media sources are not equivalent to established scientific journals in regard to topics of science.

Notable viewpoints may get articles (sources and quantity of information permitting) which may or may not receive links from (but generally not sections in) the main article for the topic (this is an editorial determination). Recognized viewpoints should have sections in the main article for the topic, whether they are the mainstream viewpoint or not (in many cases, these will be stub-sections that direct to daughter articles). Established viewpoints, when one exists for a topic, may be presented as mainstream or definitive; other viewpoints discussed in the topic's article should be considered alternative outlooks. Established viewpoints with fewer, lower-standing counter-viewpoints can be presented with more certainty than those with, for example, several recognized alternatives. When appropriate, an established viewpoint with may be able to be presented as wholly factual (the earth is an oblate spheroid, Elvis is dead), but editors are advised to err on the side of restraint when uncertain (see the use of the phrase "currently prevailing definition" in the speed of light article as an alternative).

Within reason, unusual circumstances of specific topics, viewpoints, and controversies may necessitate deviation from the guidelines.

Regarding your examples:

  • Creationism is only notable as science because that community (scientists) does not put it forth as factual, nor does it receive coverage in equivalent sources to its primary counter-viewpoints (cosmology, evolution, geology, etc.). On the other hand, within a religious context, it is certainly recognized (though not established). Thus, it does not prevent science topics from being established, but (unlike most only-notable alternatives) probably does deserve a brief mention and link as a diametrically opposed viewpoint in appropriate articles due to the volume of discussion and media traffic.
  • AIDS reappraisal is recognized. Although met with justifiable derision, the topic has shown up journals and resources that cover the mainstream topic. I don't agree with it (but that doesn't matter), and neither do most scientists, but they are talking about it, and we have to respect that. HIV/AIDS can still be presented as established because it is unquestionably the only viewpoint presented as mainstream and factual outside AIDS advocacy and research (i.e. in textbooks, other encyclopediac compilations, and government sources). The reappraisers still get their article section and daughter article, just as they do now.
  • Cold fusion has no established viewpoint. Mainstream science is certainly trending that way, but for now there is still a great deal of controversy, with proponents arguing that their results are at least possible under the theory, and everyone else asking why reproducibility seems ever so hard for them. I would give the cold fusion opponents slightly more weight, but this is not the place to be wholly definitive yet.

Whew! Hope that wasn't TL;DR! Serpent's Choice 10:29, 25 October 2006 (UTC)[reply]

Not at all too long, this is outstanding. Let's see if I can first boil it down to a soundbite and then work through the logic of applying it. Notable = merits inclusion; recognized = merits inclusion and description as significant alternative in main article on the topic; is equivalently (albeit not equally) supported to its opposition (not same number of journal articles, but perhaps articles in the same journals); established = can be stated as fact. If A is recognized, that does not mean that not-A cannot be established?
creationism (in science context) - notable (because of adherents), not recognized or established (because of extreme scarcity of scientific publications on it); "anti-creationism" is established (through multiple mainstream secondary/tertiary sources). ok.
aids reappraisal - notable and recognized (because it is covered in multiple articles by well-known biologists in mainstream science journals, same as the opposition), not established. the hiv theory of aids is established (because it is described so in multiple secondary sources, while its opposite is not).
cold fusion - notable and recognized (because it is covered in multiple articles by well-known physicists in mainstream science journals, same as the opposition), not established; and anti-cold fusion is not established either, because secondary sources are equivocal?
And if I follow the suggested treatment: notable stuff gets an article; recognized stuff gets sn article and a mention (space proportional to prominence) in main article on the topic (here: aids reappraisal under aids, and cold fusion under fusion); established stuff may be stated without qualifiers (although still needs to be sourced). Anything which is not established requires qualifiers ("as reported/claimed/stated/proposed by X, ...") in the text; anything which is established does not require qualifiers but may use them as a matter of editorial discretion. Right?ObsidianOrder 11:34, 25 October 2006 (UTC)[reply]
Right what I had in mind. Glad to see that writing in the early AM local time doesn't rot my brain too badly. I won't claim this suggestion as a panacea, but at least its a place to start. Regarding your one question ("If A is recognized, that does not mean that not-A cannot be established?"), if A is established, then not-A cannot be established. Establishment is itself a pretty broad window, ranging from "dominant mainstream outlook" all the way to "(functionally) indisputed fact". I didn't have a good breakpoint there, so I left the gradation up to case-by-case examination and editor discretion. If A is established, there can be a recognized not-A, but that means A would be closer to the "dominant mainstream outlook" than "indisputed fact" -- the wordings wil differ, of course, but we'll still present the established viewpoint strongly because it is at that point clearly the encyclopedic one. Serpent's Choice 11:58, 25 October 2006 (UTC)[reply]

Findings of Fact and Remedies

[edit]

For a good example of how not to edit a workshop page, see Wikipedia:Requests for arbitration/Rachel Marsden/Workshop. Countless hours of work and likely nearly all will be ignored. I am trying to write some proposals and findings of fact based on a review of past arbitration cases. I'm not done yet, so if it looks imbalanced you can wait for me to catch up or write your own. It is important that remedies be supported by findings of fact and that findings of fact be supported by evidence. At the moment it seems to me that the evidence focuses too much on proving your arguments about science and not enough about editors behavior. The arbitrators, ranging from a retired attorney to a college history major, are not going to decide whether the Big Bang is scientific theory or pseudoscience, but they may very well decide that editors who can't play nice together should be kicked out of the playground altogether to make room for some new faces. Thatcher131 14:46, 17 October 2006 (UTC)[reply]

Thatcher - the problem is not behavior, it is policy (clarification). Should it be Wikipedia policy to write science articles from a scientific POV or from a neutral POV? Should minority views in science (cold fusion, aids reappraisal) be given a large amount of space in their own articles, or should those articles be devoted primarily to elaborating the mainstream criticisms of such theories? What about views which are quite likely pseudoscience (time cube), are those treated differently? How do you make the distinction? Should minority views be mentioned in mainstream articles? What specific criteria of evidence should be used for determining that something is mainstream (or consensus) science, and is it appropriate to state that in an article? What specific criteria of evidence should be used for determining that something is pseudoscience, and is it appropriate to state that in an article? I think those are sufficiently general issues that they don't require people with a lot of science expertise. Also most of the personal friction between editors seems to come from the fact that they interpret the current (vague) policy in different ways, and I would thing all in good faith, not from any inherent inability to play nice. ObsidianOrder 18:03, 17 October 2006 (UTC)[reply]
Wikipedia policy addresses most of the above issues and I have at some points been involved in stating them as clear as possible. Thus I'm badly surprised to see the above statements that suggest that it's still not clear enough. It's generally believed by editors of WP:NPOV that the problematic point is undue weight, as that's hard to measure. Harald88 22:35, 19 October 2006 (UTC)[reply]
Harald - I cetainly thought I knew what the answers to the above questions about policy are; and yet, other editors (for example Pjacobi at the top of this page) interpret policy as the exact opposite. Hence the need for clarification. ObsidianOrder 23:19, 23 October 2006 (UTC)[reply]
I suspect the arbitrators will decide that the problem is not unclear policies but editors who can not work together cooperatively to follow existing policies. However, they may be amenable to making some policy clarifications. Thatcher131 21:05, 17 October 2006 (UTC)[reply]

A procedural question. I thought about adding something like this to the principles:

The ArbCom does not settle content disputes

[edit]

1) It is not the role of the Arbitration Committee to settle content disputes; in many cases, the arbitrators are not qualified to do so. More specifically, the Arbitration Committee will not issue any declarations as to whether or not the various scientific theories under discussion in this case constitute "pseudoscience"; the primary focus on this case will be user conduct.

Comment by Arbitrators:
Comment by parties:
Comment by others:

... but wasn't certain if this is appropriate or not. Certainly, deciding which cosmology is correct is beyond the arbcom; prononuncing a theory to be "pseudoscience" (which would mean an indictment of the conduct of researchers--even though Eric Lerner is a party; his off-wiki research is off-limits to the arbcom) is likely beyond the arbcom as well. Does the arbcom make the more limited ruling that WP:FRINGE or undue weight may apply to a topic, or is that beyond it as well? --EngineerScotty 21:31, 17 October 2006 (UTC)[reply]

Obviously WP:FRINGE isn't policy nor is it enforcable, as WP:FRINGE very well stresses.
Undue weight is part of WP:NPOV; thus ideally there should be ways of establishing it. Harald88 22:27, 19 October 2006 (UTC)[reply]

Findings of fact?

[edit]

How do "findings of fact" on the workshop page differ from the facts presented in evidence? --Iantresman 21:09, 17 October 2006 (UTC)[reply]

Best help is to read some past cases. A rough analogy is that the evidence page is like the evidence stage of a trial, where the attorneys present lots of facts (the facts that support their side, naturally), mixed with allegations and one-sided arguments. Like a judge or jury, the arbitrators then sift through the evidence and decide which to accept as the facts of the case. Suppose user Smith offers evidence that Thatcher131 has violated 3RR and made personal attacks. A finding of fact might be "Thatcher131 has edit warred and been uncivil." On the other hand, if the arbitrators find that Thatcher was reverting contentious POV material added by Smith and that Smith was baiting Thatcher with personal attacks of his own, the arbitrators might leave it unsaid or even come back with "Smith has edit warred and been uncivil." Which for example is why Art Carlson's evidence supports (in my opinion) findings of fact that both he and Elerner were engaged in edit warring. (The arbitrators will rewrite the workshop proposals to fit their views of the situation and eventually vote on them.)
When someone files an arbitration case there is every possibility it will come back to bite him in the butt, if his conduct contributed to the situation. Thatcher131 21:20, 17 October 2006 (UTC)[reply]
So should only arbitrators be presenting findings of fact? Natually I think all my evidence supports the statements I made, and are verifible facts. This seems somewhat biased. Likewise the other sections on the page. --Iantresman 21:30, 17 October 2006 (UTC)[reply]
Anyone can propose a finding of fact, just like anyone can propose principles or remedies. The arbitrators will pick the ones they agree with or write their own. I would like to look at the conduct of all involved editors but I started with ELerner and won't get around to the rest until later, maybe a couple of days. It would be helpful to read other cases to see how their thinking usually works. Thatcher131 21:37, 17 October 2006 (UTC)[reply]

Ian, distance yourself

[edit]

About [8] - once again, plasma cosmology doesn't mean any cosmology that recognizes plasma - that would make ScienceApologist a plasma cosmologist. Ian, please distance yourself from this kind of transparent nonsense from Tommy, or at least give him a less-than-ringing endorsement as I gave ScienceApologist here. The way this arbitration is going, you might not get another chance for a long time. Art LaPella 13:51, 18 October 2006 (UTC)[reply]

Art L.'s comments on Tommysun

[edit]

In response to [9]: I didn't mean to say that being reverted by ScienceApologist proves Tommysun is disruptive - that depends on one's opinion of ScienceApologist. I mentioned it only to explain why I only criticized Tommy's talk page behavior. I also didn't mean to say that repeating the same thing over and over again is disruptive - I've done just that when criticizing Tommysun. What is disruptive is to repeat the same thing over and over again, to be refuted over and over again, and to continue to repeat the same thing over and over again without bothering to address previous criticism. Art LaPella 02:27, 19 October 2006 (UTC)[reply]

Note: The arguments supporting the reverts of tommysun's work are "nonsense" and "irrelevant" which is obviously a POV which obviously is not amendable to reason. Tommy Mandel 15:56, 19 October 2006 (UTC)[reply]
Note note: I agree ScienceApologist's word "nonsense" could have been more illuminating, but "irrelevant" was a good reason that Tommy himself wasn't very amenable to answering. Art LaPella 21:40, 19 October 2006 (UTC)[reply]

Here is what a fact looks like

[edit]

Ladies and gentlemen, I present you with a simple sentence written by an anonomous editor which in a nutshell illustrates our case --Following is a statement found on the history page of the cosmology article.

(Redundant. Current physical cosmology does not *argue* for the big bang, it presupposes it.)

This is how the real Wikipedia works...If that is what Wikipedia is about, if that is what you want, then you know what to do.


Tommy Mandel 08:15, 19 October 2006 (UTC)[reply]

AND
"Tommysun primarily edits talk pages, as his edits to main pages are usually promptly reverted by ScienceApologist."

An admission of guilt that was removed by Art LaPella.

Tommy Mandel 15:04, 19 October 2006 (UTC)[reply]

I don't understand. What admission? What did I remove? Art LaPella 20:58, 19 October 2006 (UTC)[reply]

What Tommysun does over and over

[edit]

A. Introduce into the plasma cosmology/big bang/alternative cosmology article, reshift section, evidence that Hubble did not believe/prove/support expansion,

B. Introduce into sections on redshift that quantized redshift has been observed/confirmed.

C. Claim over and over that big bang advocates have a vested interest when they edit their opponents article.

D. Claim that Plasma Cosmology should be about Plasma cosmology.

E. Claim that big bang supporters edit their POV into alternative articles.

I was told at the beginning that over a period of time an article moves toward accuracy through the variety of postings. I have not found this to be true in all cases.

(the above is part of a comment by Tommysun)
What Tommysun does over and over is, for example, "C. Claim over and over that big bang advocates have a vested interest when they edit their opponents article" and therefore that they shouldn't be editing Plasma cosmology. We have often answered him that Wikipedia allows articles to be edited by opponents, a point he has often ignored instead of answering. That's an example of my point. Art LaPella 20:55, 19 October 2006 (UTC)[reply]
I ask again, why are the big bang proponents so concerned about Plasma to the degree that they remove evidence that does not support their POV, claiming that it is wrong/irrlevant/nonsense/belongs in some other article? And if in fact one of the editors is employed by a cosmology institute, then doesn't that employment constitute a conflict of interest? The answer I heard directly from SA was that "I am a Wikipedian" but when I asked the same question earlier on one of the lists one of the replies was "A good Wikipedian can do as he damn well pleases." I cannot accept either one of those answers.

proposal to separate criticism from article

[edit]

Using plasma cosmology as an example, the article would be written by directly involved parties. Criticisms of the article would be in a special section clearly labeled as such. And if "criticism" is a POV, then "notes"

Tommy Mandel 09:02, 19 October 2006 (UTC)[reply]

No. Please see Wikipedia:Content forking. Thatcher131 11:42, 19 October 2006 (UTC)[reply]
  • NPOV "Fairness_of_tone" suggests: "refuting opposing views as one goes along makes them look a lot worse than collecting them in an opinions-of-opponents section .. We should, instead, write articles with the tone that all positions presented are at least plausible"
  • Is this consistent with Content Forking, or is different? --Iantresman 12:06, 19 October 2006 (UTC)[reply]
Oh, I misread tommy's comment. I thought he was talking about separate articles. Content forking is the creation of separate articles for pro and con views of a topic. The usual Wikipedia solution for when editors can not work together on an article is not to create separate articles for each side but to get new editors (via RFC, third opinion, or in extreme cases, topical bans handed down by arbitration). I think there is probably no single right way to deal with criticism in articles. I think having a "criticism" section can in some cases draw undue attention to a topic, for instance in some biographies, or in small corporate articles. In science topics, I would say that it is important to fairly present the topic before criticizing it. On a fringe topic that would mean fairly setting forth the views of the topic's proponents, followed by criticism and rebuttal (adhering of course to policy on independent reliable sources, no original research, and so on). Note that my view on how a topic such as Intrinsic redshift should be treated in its own article are somewhat different than my views on how intrinsic redshift should be treated in other articles such as Big bang, Inflation theory, Hubble constant, and so on. See my proposed principles (8-11 I think). Thatcher131 12:17, 19 October 2006 (UTC)[reply]
  • No problem. If you look at how other encyclopedias treat minority views, they tend to place criticisms in a separate section towards the end. A comment in the introduction is sufficient to tell the reader that the following article is controversial and not accepted by many others. With that in mind, there is no need to turn the detailed description of the minority view into a debate because the article has already been correctly chracterised.
  • When I read an article, on for example, Ghosts, I know it's controversial from the very first sentence. Readers will not be "suckered" in, and there is no POV pushing because I already know it is controversial. The section on "Sceptical analysis" then provides the critical view.
  • Likewise this Arbitration case. We all the chance to present out points of view. Image how difficult it might be to understand them, of we all commented within each others' statements.
  • Readers are not stupid. We do not need to clarify each statement describing a minority view, with the majority view. And in most cases, Wikilinks would let readers do this if they want to. --Iantresman 13:00, 19 October 2006 (UTC)[reply]
True that reader are not stupid. Rather readers lack information about a topic and depend on Wikipedia to be a useful, up-to-date, and accurate reference work. FloNight 15:48, 19 October 2006 (UTC)[reply]
Readers lacking the opportunity to find out about minority scientific views, will lack the information which certain editors know, but have decided the reader should not have. --Iantresman 22:51, 19 October 2006 (UTC)[reply]
On this point I fully agree with Thatcher131, Iantresman and FloNight. Harald88 22:21, 19 October 2006 (UTC)[reply]
Can I ask a question? Isn't it true that an article should be primarily about what the article is said to be about? This horrific controversy exists only when the field of cosmology is considered. Clearly an article about plasma cosmology should be about plasma cosmology. By default, the article should be about what the title says it is about. The controversy of what is undue weight and what is NPOV exists here only when the two are considered together, in the article "Cosmology" for example. There is no intrinsic reason to discuss the big bang theory in the plasma cosmology article. Nor is there any apriori reason to consider both of them together in the plasma cosmology article.
Unless, of course, the plasma cosmology edit states a claim about the big bang theory. Then that becomes a different situation. However, if no suvh claim/mention is made, then it is/should be about plasma cosmology. Doesn't this make sense? Tommy Mandel 03:07, 21 October 2006 (UTC)[reply]
So it's okay to have an article about Holocaust revisionism that fails to discuss the competiting theories? How about an Elvis in the 21st Century article that talks about the places Elvis has gone and his friends and lovers in the 21st Century? You're okay with that not mentioning that Elvis is (sorry, is believed to be by everyone who has the slightest clue; sorry, by the vast majority of reputable sources) dead?--Prosfilaes 10:53, 21 October 2006 (UTC)[reply]

(back to margin) What I am struggling to say is that Plasma cosmology is not a class of theories which would require mention of all the other theories in the class as the article "cosmology" should do. PC is a particular theory and it seems to me that all of it should be presented first, and then it would be proper/honest to note the relationship to the mainstream theory. As it is, everything we try to edit in is reviewed/reverted by the big bang advocates. The result is that the article is stripped of the many nuances which make it interesting, and concepts crucial to plasma do not even get mentioned. what remains is sterile. When I first came here, plasma was defined as an electricity, and it took some doing to correct that. Even today the concept of scalability is barely mentioned. But scalability of plasma is crucial because it means one can test it in the lab and extrapolate the results to a galactic scale. Why is that constantly reverted out? Art is going to pop up and say that it is in the article, but not in the sense that it should/could be. There's a graphic in the article with a caption. The graphic is showing scalability, yet scalability is not in the caption, and they revert it out every time I try to put it in. Why?

Damn that Art LaPella! If only he wouldn't look here, I could simply say scalability has been edited out of the article! Art LaPella 19:51, 23 October 2006 (UTC)[reply]
Oh, yes: we did misdefine plasma as electricity. Correcting it took some doing only because I wanted confirmation from the scientists (I thought Tommy was probably right) and because the edit war Tommy stirred up had the page protected. Art LaPella 20:04, 23 October 2006 (UTC)[reply]

Suggestions

[edit]

I am not certain how ArbCom wants to focus the results of this RFAr -- whether they will primarily address conduct issues or whether an effort will be made to give an interpretive ruling regarding the problems with policy. If the former, I have no input here. If the latter, I would suggest a four-tiered structure. Certainly, there will be questions about which tier some theories apply to, especially during genesis of new theories or during a paradigm shift. Still, I feel such a clarficiation is a substantial improvement over the current wordings, without necessitating any actual change in policy per se.

1) Recognizing Wikipedia's goal to be a responsible and respected source of encyclopedic information, articles about scientific topics should primarily cover the mainstream scientific viewpoint.

2) When alternate scientific theories are treated in the same (or comparable) publications as the mainstream theory, those alternate theories should be discussed in their own section of the topic's article. If there is sufficient material in total, they may also warrant a daughter article. Regardless, the weight given such theories should be appropriate to their recognition in the scientific community. This is likewise the case for competing theories when no clear mainstream scientific consensus has formed.

3) When alternate scientific theories are not treated in the same (or comparable) publications as the mainstream theory, but are documented in sources that meet Wikipedia's requirements for reliable sources, they generally should not be discussed in the primary article. Individual articles for such theories can be created; their length and level of detail must respect undue weight guidelines with respect to the main article.

4) When alternate scientific theories are treated exclusively or nearly exclusively in self-published material and/or are discussed only in the context of a single proponent or sole creator, they should only be discussed within the article for their creator or proponent. If sufficient reliable sources also do not exist to support an article for that proponent, then such a theory should not be included in the encyclopedia unless further appropriate sources arise.

Serpent's Choice 10:44, 23 October 2006 (UTC)[reply]

Support pretty much all of the above. One addition: articles specifically about alternate theories should strive for a balance between describing the things that the theory claims (or its supporters claim), and describing the mainstream view/criticism. Articles should allocate substantial space to both, but no less than half should be about what the theory claims (a 60/40 ratio may be a good numeric target). ObsidianOrder 23:27, 23 October 2006 (UTC)[reply]
This "balance" criterion for articles about alternate theories is artificial. It should be up to the individual editors and should reflect the best secondary sources available about the alternate theory. Imposing an artificial ratio for how much criticism belongs in an article is to impose an external valuation of alternate theories that may or may not be justified. Certainly some articles should have more than half of the article devoted to criticisms. Some articles should have well-less than one half of the article devoted to criticism. It is an article-by-article determination and there is no panacea. --ScienceApologist 12:34, 24 October 2006 (UTC)[reply]
"best secondary sources" - what exactly policy says that primary sources are unacceptable? "impose an external valuation of alternate theories" - no, although devoting most of the article to criticism certainly does that. "some articles should have more than half of the article devoted to criticisms" - that would not be sticking to the topic, which is the theory, not the criticism of the theory (also relevant, also allocated substantial space if there is substantial criticism, but not the whole article - if you want that, that is a different article). ObsidianOrder 15:26, 24 October 2006 (UTC)[reply]
Complete agreement. There is a huge variance in the quality, substance, and scientific value of minority-held theories. This is especially true in the more esoteric subfields of physics. Tightening the loopholes in NPOV to prevent abuse of the system is of benefit to the encyclopedia and its readers. Mandating certain ratios of content seems more likely to devolve into wikilawyering, and would do a disservice to legitimate minority-held opinions from particularly esoteric subfields of physics (string theory variants, some models of vector gravity, and so forth) that deserve less than the magic ratio of criticism while providing a shield for unmitigated crackpottery (perpetual motion, and so on) that deserves more. Serpent's Choice 14:54, 24 October 2006 (UTC)[reply]
SC - "huge variance in the quality" - yes, but we cannot judge that ourselves, and trying to determine it from sources is somewhat problematic. For theories which are mainstream, it is trivial (pick seveal recent textbooks in the field and check what they say). For theories which are not mainstream, it is effectively impossible (do a survey??). "Mandating certain ratios of content" - ok, agreed, ratios is probably stupid, I would just say "substantial space should be allocated to both" and leave it at that. "that deserve less than the magic ratio of criticism" - fair enough; please note that the ratio specified the maximum of criticism, not the desired number. The only rationale for my proposal is that an article about X should be, well, primarily about X. If we want an article about "criticism of X" that is a different article. If there is significant criticism, that should take significant space in the article about X, but not the whole article. It is akin to having the article about George W. Bush be primarily devoted to "why Bush is a bad president": perhaps a widely held opinion, but it is not appropriate as the main content of that article (and indeed, there is a separate article for it, here). ObsidianOrder 15:26, 24 October 2006 (UTC)[reply]
Two observations. First, I think that policy provides the tools needed to have at least some idea what we're dealing with regarding minority-held theories. Everyone's favorite (or not) policy section -- undue weight -- says that we should represent viewpoints "in proportion to the prominence of each". Because Wikipedia doesn't judge truth, that "prominence" must mean representation in reliable, context-appropriate sources. If a theory isn't the mainstream one, but people in the field have at least talked about it somewhere, that's a large step up from a theory documented only on its creator's website, or even in the journal of a discipline that is totally unrelated. Does that mean we give little if any space to some amateur cosmologist who can't get his theory printed in anything that isn't self-published? Of course, that's what WP:V is about. But, second, I don't think that any ratios are a good idea, even a numberless "significant space" one. We have to make sure that we don't paint ourselves into a corner regarding theories that have been discounted as demonstrably wrong but still have adherents (archetypical example: Flat Earth). If the only reliable sources for an article demonstrate the universal condemnation of a theory, then our article on that topic should of course still discuss its tenets before presenting the criticisms, but a neutral point of view doesn't mean a disingenuous one. Serpent's Choice 15:58, 24 October 2006 (UTC)[reply]

New proposal

[edit]

See Wikipedia:Notability (science), which has this RFarb as one of its inspirations. --EngineerScotty 19:43, 30 October 2006 (UTC)[reply]

I'm confused about why we are here

[edit]

This recent post by Science Apologist sums up my general thoughts on this case pretty well:

Yes. Pseudoscience that is notable enough can have its own article. Criticism of it is included from the scientific community, skeptical societies, and verifiable contradictions to mainstream scientific thought. Pseudoscience is left out of mainstream articles unless there is some way it has established enough notability to warrant mention. The relative weight and notability of the pseudoscience is an editorial decision discussed in WP:NPOV#Pseudoscience, WP:NPOV#Undue weight, and WP:FRINGE to be made by people familiar enough with the subjects to know whether something is considered generally important or not. --ScienceApologist 21:19, 2 November 2006 (UTC)

except that I would include "Pseudoscience that is notable enough can have its own article that fairly and accurately describes it using favorable works from its proponents. (And still including criticism from mainstream science.

So in a very real sense I am confused about why we are here. There is little to no evidence of Eric, Ian or others disrupting mainstream science articles by forcing in fringe topics in violation of NPOV and undue weight. All the drama seems focused on fringe topics. Other than Eric Lerner (who I proposed has edited disruptively and who has a conflict of interest per Fred) most of the contentious editing seems to be on the side of mainstream science proponents trying to marginalize fringe science topics in their own articles, which seems to violate Science Apologist's own principle above.

I guess if I had a vote I would endorse the principle outlined above, which seems to be encompassed on the proposed decision page by principles 3A, 4A, 5, 11, 12 and 13, and maybe warn the mainstream scientists not to be so tendentious in editing fringe topics, but to remain vigilant in how fringe topics are addressed in mainstream articles. Thatcher131 15:53, 3 November 2006 (UTC)[reply]

  • This case is not about the representation of pseudoscience. As I've said from the outset, it is about the misrepresentation of minority peer-reviewed scientific views as either non-notable or pseudoscience using a variety of inappropriate tactics that I describe in my evidence.
  • The description of pseudoscience is a side issue, and I really don't see any problems as long as it is in line with WP:V, WP:RS and WP:NOR. --Iantresman 16:52, 3 November 2006 (UTC)[reply]

Solutions

[edit]

Arbcom can decide upon what to do about enforcing NPOV policy. The most sensible thing to do will be to decide which pseudoscientific subjects (eg pseudoscience art, chiropractic art, scientology art, etc) to police properly (mediate thoroughtly) and then go ahead and do it. Its pretty simple. If there is a good mediator/s there to decide when people are crossing the line then that will help a lot. Incivility punishments (rigorous blocking) will most likely help though they are not nice to work with. Practical steps! Simply prioritize the articles and assign accordingly. KrishnaVindaloo 17:56, 24 November 2006 (UTC)[reply]

You are imagining we are much more organized then we are. Fred Bauder 18:57, 24 November 2006 (UTC)[reply]
OK Fred, perhaps I overstate how easy it will be. In fact there is no perfect solution. But as long as there is a decision to place some authorities in key problem areas, there will be some sort of solution. I appreciate perfection is impossible and I'm not expecting miracles. But some groups of editors really do need to be reminded, quite regularly and sharply, about reasonable and constructive NPOV application. KrishnaVindaloo 18:15, 25 November 2006 (UTC)[reply]
  1. ^ 1
  2. ^ 1
  3. ^ 1
  4. ^ 1
  5. ^ 1
  6. ^ 1
  7. ^ 1
  8. ^ 1
  9. ^ 1
  10. ^ 1
  11. ^ 1
  12. ^ 1
  13. ^ 1
  14. ^ 1
  15. ^ 1
  16. ^ 1
  17. ^ 1