Polis Experience Report: Should Same-Sex Marriage Be Legalized?

2023-04-28 Debate on whether same-sex marriage should be legalized has gained momentum on Japanese-language Twitter

• The author has prepared a Polis on this issue
• Seed opinions were prepared in Japanese and spread through Twitter in Japanese.
• In the end, 541 people participated in the voting, with a total of 14290 votes cast.
• Polis’ visualization algorithm divided the voters into two groups

  • 

  • Detailed report: https://pol.is/report/r7suad4c8dxcprhxbfdfh

• In this experiment the seed opinion of Polis was created by [GPT-4
• In the course of the experiment, it was argued that there are many questions that should not be associated with the subject matter
  • The claim itself was subdivided and made subject to vote as the opinion of Polis.
  • Interesting results.

Commonly supported opinion

.

• 
  • The poll results indicate that many people are in favor of the legalization of same-sex marriage and the rights of same-sex couples.
    • Same-sex marriage should be legalized: 73% agreement, 13% disagreement, 13% abstention
    • Same-sex couples have the right to adopt: 73% agree, 9% disagree, 16% abstain
  • It also indicates that a minority of people believe that homosexuality and same-sex marriage are contrary to their religious beliefs.
    • Legalization of same-sex marriage goes against religious beliefs: 5% agree, 72% disagree, 21% abstain
• - Here's one that is both highly supported and suggests a next action - A new framework is needed that encompasses the family as a social partner and community, not just the old marriage: 78% agreement, 9% disagreement, 11% abstention. - Marriage as social proof and marriage as taxation and social security have their own issues to be considered in different layers. : 74% agreed, 11% disagreed, 13% abstained. - I think it would be better to elaborate on this point and look for more supportive opinions

disagreement between opinion groups

.

• 
  • We find that Group A tends to have generally more conservative views on issues related to same-sex marriage and same-sex couples than Group B.
    • Separation of social proof from taxation and social security:.
      • A’s agreement rate is 86%, higher than B’s 69%. This indicates that A feels the need for more active consideration of this issue.
    • Opinion that a position opposing same-sex marriage is discriminatory:.
      • A has an 81% disagreement rate, while B has a majority (65%) in favor. This indicates that A tends not to consider its position against same-sex marriage to be discriminatory.
    • Granting same-sex couples the same legal rights as opposite-sex couples:.
      • A has a 61% disagreement rate, while B has a majority (80%) in favor. This indicates that A is more likely to believe that same-sex couples should not be given the same legal rights as heterosexual couples.
    • Recognition of Same-Sex Marriage and International Cooperation:.
      • A has a 67% disagreement rate, while B has a majority (69%) in favor. This indicates that A tends to disagree with the opinion that recognition of same-sex marriage is important for international cooperation.
    • Same-sex marriage symbolizes social progress:.
      • A has a 73% disagreement rate, while B has a majority (56%) in favor. This indicates that A tends not to consider same-sex marriage as a symbol of social progress.
• 
• 
  • Group B tends to have generally more progressive views on issues related to same-sex marriage and same-sex couples than Group A.
    • Granting same-sex couples the same legal rights as opposite-sex couples: majority in favor as noted above
    • Legalization of same-sex marriage:.
      • B has a 92% agreement rate, while A has 49% disagreement. This indicates that B strongly favors the legalization of same-sex marriage.
    • Opinion that the position against same-sex marriage is discriminatory: as stated above, the majority is in favor of it.
    • Recognition of same-sex marriage and international cooperation: as noted above, the majority is in favor.
    • Same-sex marriage improves social equality for same-sex couples:.
      • B’s agreement rate was 89%; A weakly disagreed with the issue with 22% agreeing, 44% disagreeing, and 33% abstaining. This indicates that B believes that same-sex marriage will improve social equality for same-sex couples.
• 

how did you start this Polis

.

• A friend told me about an exciting Tweet that could be a Polis story.
  • Raw log: The impetus for “Polis: Same-Sex Marriage”.
• I read the replies to that tweet and found it harder to do than the previous experiment. - Polis Experience Report: Should we investigate the causes of terrorism?
  • What’s the difference?
    • The last time I did this, I started with a log of two people calmly discussing each other’s opinions.
      • This was easy to read what was a divided opinion for and against
      • 
    • This time it was a lot of people writing their thoughts in response to one person’s call
      • I think most people wrote their own ideas without reading what others had to say.
      • 
      • Now that I think about it, it would have been better to focus on the second and subsequent replies to the call, rather than the direct replies to the call.
• Based on this experience, it may be better to have a small group discussion first and then extract seed opinions, rather than using buzz on social networking sites directly as a data source
• In this experiment, we let GPT-4 make the seed opinion.
  • This was better than I expected.

Seed opinion prepared by GPT-4

.

• Prompt: please make a list of simple, Yes/No answerable opinions to be voted on regarding same-sex marriage.
• - 1: Should same-sex marriage be legalized? (Yes/No) - 2: Do you believe that same-sex couples marrying would have a negative impact on traditional family values? (Yes/No) - 3: Do you think same-sex marriage improves social equality? (Yes/No) - ...
• Modified wording to conform to display in Polis.
• Some sentences were split to clarify meaning. Example.
  • INPUT:
    • GPT4: Do you believe that same-sex couples have the right to use adoption or surrogacy?
  • OUTPUT:
    • Same-sex couples have the right to adopt.
    • Same-sex couples have the right to use surrogacy.
  • RESULT:
    • 
    • Poll results show significant differences in responses between adoption and surrogacy
      • Writing multiple words in parallel often reflects the author’s assumption that the words are similar
      • I think putting each on a separate agenda would increase the odds of unexpected discoveries.
• Seed Opinion by GPT-4: Better than expected!
  • Able to list “different axes” to the theme
  • GPT-4 has a penalty parameter that prevents repeating similar content
    • Frequency and presence penalties: OpenAI API
  • I guess this works well to create diversity of opinion.
• relevance - three-person conference method - This could work well too.

[discussion of too many irrelevant questions

.

The following user-submitted comments were received at this time

• Many questions are ambiguous and should not be associated

In Polis, you can select “don’t know/neither” if you think the opinion is ambiguous.

The concept of “questions that should not be associated” is interesting. Is there a “question that should not be associated”?

• Suppose an opinion completely unrelated to the theme is posted, e.g., “Cats are cuter than dogs.
• The ratio of approval or disapproval for this opinion does not change in any opinion group, so it does not appear in the “Opinions that characterize the group” section
• It could be in the majority opinion.
• If a person posts an opinion and N other participants vote for it, we can consider that N social costs are incurred for that opinion
  • Opinion contributors can show their opinions to N people and get a response, and may feel a personal benefit from it
  • When personal gain drives the posting of low quality opinions, the costs outweigh the social benefits. - Tragedy of Common Land
  • This could be solved with incentive design.
    • For example, in order to pitch an opinion to N people, the mechanism is that I first vote for N cases, accumulate credits, and then pay for them.
    • The mechanism could be that if there is not enough money on hand, it becomes a debt, and the next debt cannot be incurred until the debt is eliminated through a subsequent ballot initiative.
    • In the latter, anyone can post one opinion.
      • Problem of too many Polis questions has been pointed out by several people, so it might be better to include such a mechanism

Opinions that one person considers “questions that should not be associated” may actually be inconvenient for that person

• GPT-4 suggested the following possibilities for this matter
  • That opinion is contrary to their position and beliefs.
  • association weakens the opinion or position they are advocating.
  • Fear that the associated questions will distract from the focus of the discussion and prevent you from fully communicating your point.
  • Trying to defend their position and opinions by arguing that inconvenient questions should not be relevant.

The following replies were made on Twitter regarding this matter

ft28 felt that the discussion of same-sex marriage and its legalization should be separated. Other adoption and use of surrogate mothers need not be associated with same-sex marriage. In general, the questions themselves have many meaningless and irrelevant associations and are not meaningfully distributed. Discussion should focus on improving the status of de facto marriage and the burden on beneficiaries The current system, which rewards family registers and genealogical survival, is not bad.

• nishio I think we should each vote on this as an opinion!

As a result of voting as an opinion

• 29: “The use of surrogate mothers need not be associated with same-sex marriage” garnered 76% agreement.
• 28: A majority of 58% also agree on “adoption does not need to be associated with same-sex marriage”
• On the other hand, 27: “Discussion of same-sex marriage and its legalization should be separated” and 30: “Discussion should be focused on improving the status of de facto marriage and the burden on beneficiaries” are opposed by a majority
  • Moreover, the ratio of approval or disapproval is inverted according to opinion groups.
  • 31: “The current system that rewards family registers and genealogical continuity is not bad” is almost a three-way tie between approval and disapproval abstention.
    • This one, like the other two, has the pros and cons reversed depending on opinion groups.
  • So the suggestions in 27 and 30 to “separate or narrow the topic” were favorable to those in Group A because “being associated weakens the opinion or position they are advocating,” and conversely, they were unfavorable to those in Group B!

As these results show, even when some claim it is an irrelevant topic, it often fails to reach consensus

• Observing diverse perspectives leads to a better understanding of things.
• Don’t be too hasty in deleting opinions.

This idea that “even if some people think it doesn’t matter, we don’t know if it really doesn’t matter” can be applied to the operator

• Even if the moderators don’t think it’s relevant, they don’t know if it really isn’t.
• I have opinion moderation turned off in these experiments.
  • On the other hand, it also leads to an increase in low quality and duplicated opinions - Problem of too many Polis questions
• A good way to do this is still in the exploration stage, but I think there is a good chance that GPT-4 or other LLMs could be useful.

---

This page is auto-translated from /nishio/Polis体験レポート:同性婚を合法化すべきか using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.