2021-05-04 An experiment to pick up keywords from the most recent one-day update in Scrapboxâs Project A and present pages containing those keywords from Project B.
- In this case, A is this public project and B is a private project of about 400 pages
If it seems to have a good effect, run it regularly once a day.
context - Diary of 2021-05-04 - About Scrapbox private to public transfer
What we did.
- Get â publicâs most recently updated pages
- â
Extract keywords from its content
- Just the one that is explicitly bracketed for now.
- You can get it from the API.
- â
Find and present â
from different sources that contain those keywords.
- There are two ways to search on your own after exporting to JSON or by hitting the search API.
- This time, the search API
- How to present
- Create a page in private
- Because the search results of a private project cannot be made public
- There is an option to send a DM in Slack.
- â this time I printed it and pasted it into Scrapbox by hand
- Create a page in private
result
- (of) good appearance
- I thought it was a private memo repository with not much information, but there were some surprisingly interesting pages.
- I got some hits from the experiment, like the one where I transcribed a recording of a video conference.
- 105 keywords extracted from 24 hours of activity
- The number of hits found in the search is 11.
- The search API yields the hit rows.
- There are some not-so-good keywords.
- Research, Organization, Scrapbox, Unexplored Jr.
- Too many hits.
- It is better to display only the information âmany hitsâ if the number of hits exceeds a certain level.
- In the original project, it is only connected if both pages are bracketed, which is not a large number.
- Three of the four cases were bracketed.
- So, if you have a lot of search results, you can only show the number of results instead of the number of results, and bracket that search term instead.
- Always do search keyword bracketing.
- Rather, âIs there already a link?â and if so, is it better not to search?
impressions
- relatively good
- There are a number of other projects that seem to have a lot of interesting content dead in the water.
- I want to have more than one target project.
additional experiment
-
I was thinking mainly of my public/private collaboration, so I couldnât publish the results because one of them was private.
-
I realized I could do both public.
- And with the current version, which uses search instead of export, you can target other peopleâs projects without any permission.
-
interest :
-
/rashitamemo/interesting(query): 24 hits
-
/rashitamemo/found(query): 88 hits
-
/rashitamemo/noticed(query): 10 hits
-
/rashitamemo/spontaneous(query): 11 hits
-
/rashitamemo/intellectual-production(query): 100 hits Puppeteer:
-
/rashitamemo/Puppeteer(query): 1 hits
-
- puppeteer
- GoogleChrome/puppeteer: Headless Chrome Node API
-
Puppeteer is a Node library which provides a high-level API to control headless Chrome or Chromium over
- [https://codeburst.io/a-guide-to-automating-scraping-the-web-with-javascript-chrome-puppeteer-node-js-b18efb9e9921
- A Guide to Automating & Scraping the Web with JavaScript (Chrome + Puppeteer + Node JS)] Heroku:
-
/rashitamemo/Heroku(query): 1 hits
-
[/rashitamemo/Headless Chrome](https://scrapbox.io/rashitamemo/Headless Chrome)([query https://scrapbox.io/rashitamemo/search/page?q=Headless Chrome]): 1 hits
-
- GoogleChrome/puppeteer: Headless Chrome Node API
-
Puppeteer is a Node library which provides a high-level API to control headless Chrome or Chromium over
- It can also be configured to use full (non-headless) Chrome or Chromium.
- It is thought to be a headless Chrome that can be operated from the command line.
- [https://codeburst.io/a-guide-to-automating-scraping-the-web-with-javascript-chrome-puppeteer-node-js-b18efb9e9921
- A Guide to Automating & Scraping the Web with JavaScript (Chrome + Puppeteer + Node JS)]
-
research :
-
/rashitamemo/research(query): 100 hits MVP:
-
/rashitamemo/MVP(query): 1 hits
- Blog 2.0 :
- fbclid=IwAR3LXwnspP4W9R79YvAjlC760Q62f_kgBMVpUmRP9WmYok8_xWEMVLkjyZM]
-
[/rashitamemo/tomatology culture](https://scrapbox.io/rashitamemo/tomatology culture)(query): 1 hits
-
campfire (usually large, and for gathering and singing, etc.) :
-
/rashitamemo/campfire(query): 1 hits
- June 2019 Idea Line :
- [I want to spread more people to use the most powerful time management technique that saved my life, âTask Shoot Time Techniqueâ - CAMPFIRE (Campfire) https://camp-fire.jp/projects/view/162572
- June 2019 Idea Line :
-
/rashitamemo/gated-community(query): 3 hits
- gated community :
- gated community
-
A gated community is a residential area with a gate (gate) and a wall around the perimeter to restrict access to the property by non-residents, thereby preventing through traffic and improving security.
- The concept of a gated community itself is not new; there have been concessions and U.S. military houses in the past, and it is merely a redefinition of the term. In Japan, it is also referred to as a gated town1 or gated community2. `
- Walled Garden Model :
- â gated community
- gated content :
- â gated community
-
/rashitamemo/organization(query): 84 hits
-
decline :
-
/rashitamemo/decline(query): 8 hits
-
/rashitamemo/exclusion(query): 14 hits Scrapbox:
-
/rashitamemo/Scrapbox(query): 100 hits
Updated to account for the case where the output project and the project to be searched are different.
-
The
[1]
in the quote from rushitamemo has become a search keyword. -
Should it be
~``? I fixed it by hand for now.
-
The output of a machine should not be fixed by hand and maintained for a long period of time.
- Iâm only doing it now because itâs an experiment.
- Itâs confusing the links for this project, so Iâll eventually kick it out to gist or something.
- Hopefully, if nothing is done, it will be overwritten after a day and disappear.
- Any useful links should be noted on a separate page
- Iâm only doing it now because itâs an experiment.
-
interest :
-
/shokai/interesting(query: 1 hits)
-
- (Do we need to be funny about how we do things?)
-
/shokai/found(query: 25 hits)
-
/shokai/notice(query: 3 hits)
-
[/shokai/not a warehouse where dead texts are kept](https://scrapbox.io/shokai/not a warehouse where dead texts are kept):.
- Itâs hard to realize, but itâs a complete waste of time and effort, because the tedious procedure somehow gives you a sense of doing your job and a mysterious sense of satisfaction.
-
- I noticed that there is also a store in Joynas, Yokohama, and I have been going there about once every two weeks since then.
-
- I realized that I had no motivation to create libraries for devices I didnât use, and I resolved to only create things I would use from then on. Heroku:
-
/shokai/Heroku(query: 39 hits) Headless Chrome:
-
[/shokai/Headless Chrome](https://scrapbox.io/shokai/Headless Chrome)([query https://scrapbox.io/shokai/search/page?q=Headless Chrome]: 1 hits)
-
- Headless Chrome library was created and got 2000stars by yujiosaka.
- https://speakerdeck.com/yujiosaka/hetudoresuchromedekurorawozuo-tutahou-falsehua-1
- https://github.com/yujiosaka/headless-chrome-crawler
-
research :
-
/shokai/research(query: 26 hits) 1:
-
/shokai/organization(query: 14 hits)
-
decline :
-
/shokai/decline(query: 1 hits)
-
[/shokai/Mob Programming](https://scrapbox.io/shokai/Mob Programming):
-
/shokai/exclusion(query: 7 hits)
-
[/shokai/Internet language can be rough because of the short term nature of the recommendation service](https://scrapbox.io/shokai/Internet language can be rough because of the short term nature of the recommendation service):: [/shokai/Internet language can be rough because of the short term nature of the recommendation service](https://scrapbox.io/shokai/Internet language can be rough because of the short term nature of the recommendation service). - Eliminate all features that stimulate approval.
-
[/shokai/eliminate all functions that lead to the stimulation of the need for approval](https://scrapbox.io/shokai/eliminate all functions that lead to the stimulation of the need for approval):.
- Eliminate all features that stimulate approval.
-
[/shokai/wiki never works if you use the URL of the page as an ID](https://scrapbox.io/shokai/wiki never works if you use the URL of the page as an ID):.
- I want to eliminate all the elements that would spawn a lackey/administrator within WiKi and have to ask before I edit.
-
/shokai/press-release:: [/shokai/press-release
- This and the idea of [Eliminate all features that stimulate approval.
-
[/shokai/I lose my reason when I look at a human face](https://scrapbox.io/shokai/I lose my reason when I look at a human face):.
- I use Scrapbox to eliminate those elements that are obscured by the appearance of real human faces.
-
[/shokai/want to reduce elements that block thinking](https://scrapbox.io/shokai/want to reduce elements that block thinking):: [/shokai/want to reduce elements that block thinking
- Like most editors, Iâve eliminated switching between edit and view modes because itâs only a brake on my thinking, and if there are other toolbars for text decoration, I spend more time fiddling with the font size than writing text.
- It is thoroughly eliminated because the time is more and the acceleration is lost.
-
[/shokai/media whose memory is reset every time](https://scrapbox.io/shokai/media whose memory is reset every time):: [/shokai/media whose memory is reset every time
-
/shokai/Scrapbox(query: 100 hits)
-
[/shokai/Hatena Diary](https://scrapbox.io/shokai/Hatena Diary)(query: 1 hits)
-
[/shokai/Internet language can be rough because of the short term nature of the recommendation service](https://scrapbox.io/shokai/Internet language can be rough because of the short term nature of the recommendation service):: [/shokai/Internet language can be rough because of the short term nature of the recommendation service](https://scrapbox.io/shokai/Internet language can be rough because of the short term nature of the recommendation service).
- I think the exchange of opinions should be done on Hatena Diary or Hatena Blog. dotenv:
-
/shokai/dotenv(query: 2 hits)
-
- dotenv
- http://npmjs.org/packages/dotenv
- Can be loaded with
require('dotenv').load()
.
-
- dotenvify
- https://www.npmjs.com/package/dotenvify
- Transform of Browserify that executes dotenv just before executing [envify
$ browserify -t dotenvify src/index.js -o build/index.js
- When
.env
is missing, dotenv gives warnings - It can be suppressed with the silent option, but dotenvify does not have the ability to pass options to dotenv
- Added functionality: https://github.com/shokai/dotenvify/tree/dotenv-options
$ npm i shokai/dotenvify#dotenv-options -save
$ browserify -t [ dotenvify --silent --path local.env ] src/index.js -o build/index.js
Secure:
-
/shokai/Secure(query: 1 hits)
-
Github to Scrapbox:: bookmarklet
- Use secure protocol instead of http #79
- [https://github.com/shokai/tw/pull/79 Use secure protocol instead of http by sachin21 · Pull Request HttpOnly:
- Use secure protocol instead of http #79
-
/shokai/HttpOnly(query: 2 hits)
-
[/shokai/Heroku to do Socket.IO](https://scrapbox.io/shokai/Heroku to do Socket.IO):.
- Version=1; Expires=Tue, 15-Nov-2016 11:51:24 GMT; Max-Age=86400; Domain=staging.scrapbox.io; Path=/; HttpOnly
-
- Version=1; Expires=Tue, 15-Nov-2016 11:51:24 GMT; Max-Age=86400; Domain=staging.scrapbox.io; Path=/; HttpOnly requests:
-
/shokai/requests(query: 1 hits)
-
[/shokai/login with Passport to GHE operated with ole SSL certificate](https://scrapbox.io/shokai/login with Passport to GHE operated with ole SSL certificate):: [/shokai/login with Passport to GHE operated with ole SSL certificate](https://scrapbox.io/shokai/login with Passport to GHE operated with ole SSL certificate):: [/shokai/login with Passport to GHE operated with ole SSL certificate
- [Adding ability to specify an agent for OAuth2 requests by PhilipSkinner · Pull Request #324 · ciaranj
impressions
- Itâs nice to see other peopleâs presentations on technical keywords that Iâm interested in and have written down.
- We all think we are using the same Japanese, but there are differences in the choice of words we use to express our thoughts.
- In my project, I use the word âexclusionâ to refer to the removal of a person from a group of people (this is not my word either, but something I wrote down after seeing other peopleâs expressions).
- On the other hand, shokai is used to refer to âremoving harmful features or functions from software.
- By comparison Notice the difference.
- If you look at the keywords displayed and tweak the page in various ways, of course it will be an âupdated pageâ tomorrow, and the same thing will be displayed.
- This must be boring.
- Remember what you put out once and donât put it out for a week to a month, and so on.
source https://github.com/nishio/scbot/blob/0.2/scbio.py#L113
This page is auto-translated from /nishio/è€æ°ăźăăăžă§ăŻăéăźăȘăłăŻăèŠă€ăă using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. Iâm very happy to spread my thought to non-Japanese readers.