Python Programming & other inferior programming languages thread

Discussion in 'The Mainboard' started by Walt Disney, Jul 15, 2015.

  1. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

  2. TDCD

    TDCD Handling the Fisher account
    Donor
    Florida State SeminolesConnecticut HuskiesBoston Red SoxNew England Patriots

    Looking for something to learn in my spare time too. I do all node at work and wanna try something non web related

    Anyone do any image recognition stuff? Thought it might be fun to learn.
     
  3. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

    not sure if we're talking about the same thing but i think any kind of machine learning like that requires a ton of time feeding it data

    a dude at my school wrote some image recognition software for his year-long senior project and he said he spent 36 hours on data entry alone, and it still had a long way to go
     
  4. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    At a conference I went to for Big Data with AWS, they recommend you feed billions of data points, if possible, for their machine learning features.
     
  5. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt


    I back up to AWS :gocho:
     
  6. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    To S3?
     
  7. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    si si
     
  8. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Does S3 have a free-tier amount even after 12 months?
     
  9. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    I think so?

    at this point AWS is already charging me for shit so I just roll with it.
     
  10. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Yeah I thought about writing some simple console application to push stuff to my AWS Account in S3, but it seemed cheaper to just roll with Google Drive.
     
    colonelrascals likes this.
  11. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    that is exactly what I did, now I'm too lazy to go back
     
  12. TDCD

    TDCD Handling the Fisher account
    Donor
    Florida State SeminolesConnecticut HuskiesBoston Red SoxNew England Patriots

    Bootcamp bubble appears to be bursting. Mine announced they're shutting down yesterday
     
    leroi and colonelrascals like this.
  13. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Damn, what company? There are a fuckload of them and these online only ones are popping up too. There are plenty of free online programs, not sure why anyone would not take advantage.
     
    #1013 Pile Driving Miss Daisy, Jul 13, 2017
    Last edited: Jul 13, 2017
  14. TDCD

    TDCD Handling the Fisher account
    Donor
    Florida State SeminolesConnecticut HuskiesBoston Red SoxNew England Patriots

    I went to dev bootcamp
     
  15. Shiggityshwo

    Shiggityshwo Well-Known Member
    Donor TMB OG
    North Carolina TarheelsAtlanta BravesAtlanta UnitedUnited States Men's National Soccer Team

    mine keeps expanding, they've got a nice niche here in Nashville
     
    Walt Disney likes this.
  16. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    can we be coworkers
    #NashvilleBroz
     
    Shiggityshwo likes this.
  17. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    where'd u go?
     
  18. Shiggityshwo

    Shiggityshwo Well-Known Member
    Donor TMB OG
    North Carolina TarheelsAtlanta BravesAtlanta UnitedUnited States Men's National Soccer Team

    Nashville Software School
     
  19. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    So the Iron Yard, arguably the most well known coding school, announced today that it's closing all of its campuses. At my now previous job we had hired several people who graduated from the school in Atlanta, most of them were good since they had all had a genuine interest and enjoyed programming. We had also hired one of the culture directors from the campus to come work with us. She mentioned that it sounds like they had a big problem with city/local codes and licenses/regulations in most of the areas they opened, but I'm not sure how that affected their business since they charged quite a bit for their semester long program. This also comes right on the heels of Dev Bootcamp, which was one of the first coding schools, shutting down but for different reasons.

    My former coworker, who is a senior Architect, and I were talking and he mentioned (which I've heard on NPR several times) that although there is a huge need for programmers (in almost a blue-collar kind of a sense), the biggest need is for more senior level and experienced programmers. The bootcamps unfortunately can't quite address that problem since such experience is only learned by having employment for a few years, but hopefully the amount of people entering the workforce will remain and gain the skills that are needed for these jobs. Not quite sure what point I'm trying to make, but it was interesting reflecting how these schools have popped up all over the place in just the past 5 years.
     
  20. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

    It could just be confirmation bias because I went a different route and I'm happy with it, but when I was figuring out what I wanted to do, I was skeptical about being able to come out with career-ready skills in 3 months or whatever. It's worked for some people but it seemed like too much of a risk.
     
  21. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

    Like, I'm sitting in my testing class right now, working on dependency injection. Is it possible to even reference this stuff when you're going from zero to programmer in 3 months? Or design patterns? Obviously people have been self-taught in this field forever so :idk:
     
  22. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Even though I studied CS for a bit in college, I don't have a degree in it. When I entered this area only a few years ago, I pretty much self taught myself through free online courses and random tutorials. Bootcamps are good, but they do have a huge focus on exclusively javascript and not as much architecture at the backend level (which like you said takes more than 3 months to learn). You don't have the freedom to fully explore what interests you the most, but the Iron Yard was generally very good at giving you a rigorous, challenging, and certainly rewarding experience. Like I mentioned, those who really love programming are going to generally be fine as they'll have the drive to continue to learn even when thrown into the fire.
     
    Joe_Pesci likes this.
  23. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    I teach for The Iron Yard
     
  24. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    this was a big problem. I'm a little drunk but I can give my :twocents: tomorrow
     
  25. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    So uh, you want to tell us what's going on?:popcorn:
     
    Joe_Pesci likes this.
  26. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

     
  27. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    tag me tomorrow so I don't forget.
     
  28. kentucky_dawg

    kentucky_dawg Fan of: Georgia
    Donor
    Georgia Bulldogs

    I think showing that you understand core concepts like these in an interview is very good leverage to landing an entry level position at a company as maybe a junior dev.
     
    Joe_Pesci likes this.
  29. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    It also depends on what your company's needs are. My previous company did 99% web development so we had a very specific need for front-end JS/React/Angular devs and backend that was increasingly becoming focused on using Node, less so how much you understand about sorting algorithms, binary search trees, tries, etc.. This is why I mentioned that TIY was fulfilling a need for us, most of the devs who were successful continued their learning and became familiar with design patterns that stretch across technologies
     
    kentucky_dawg likes this.
  30. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    1. People who can effectively teach others to program well enough to secure professional employment may have better options than teaching open-enrollment entry-level programming classes.

    2. Because of that, and for any of a number of other reasons, we may generally be overestimating how effectively these kinds of schools are educating people.

    3. It's an open secret that hiring in the technology industry isn't based on aptitude, but rather on credentials and hazing rituals. Something like 3 decades of cultural development have trained developers that the most important risk they face in hiring is accepting someone under qualified, rather than passing on someone who is in fact qualified.


    I was fortunate enough to be on an amazing team at my TIY, but instructors and culture varies at every campus. I don't now how many bad eggs there were but some campuses had a reputation of being sub par. Unfortunitly there wasn't much you could do about this because of problem #1, this meant a closing of campus and loss of profit. The investors that backed us weren't making the returns they'd hoped and they pull out, our old CEO had made an attempt to rebuy the company, but it failed.
     
  31. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Pretty sure I read almost this same comment on Hacker News. :blount:

    So in your opinion, the biggest problem was not finding and retaining enough qualified instructors? Like, as in, the campuses literally could not operate well because of such a shortage? A couple of the comments on the Hacker News posting (may have been you) stated that the instructors were paid very poorly which I find crazy, you literally cannot penny-pench in this one area. You'd think a coding bootcamp would understand the value of someone with such a skillset considering the market is absolutely bananas right now for devs, I got an enormous raise to take my current job.
     
    colonelrascals likes this.
  32. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    :awesomeface:
     
    Room 15 likes this.
  33. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    Yes, and because of this they had to find ways to make due with less -than qualified instructors.

    Poorly in comparison to the jobs I could get? Yeah.

    I have a fantastic work life balance though, and all things considered make more than enough money.

    At a campus level they most certainly understand the value. As we expanded so quickly I think corporately we lost sight of that.
     
  34. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    What languages will be the most popular in 5-10 years

    I really feel like Elixir is about to come on strong but it's always a risk with these new hot languages

    I know I'm learning functional programming with elixir so that will always be helpful
     
    colonelrascals likes this.
  35. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    [​IMG]
     
  36. colonelrascals

    colonelrascals Mayonnaise-colored Benz, I push miracle whips
    Donor
    Texas Tech Red RaidersHouston AstrosHouston RocketsTennisTottenham HotspurTexas Tech Red Raiders alt

    anyone use mint ?

    kali?
     
    #1037 colonelrascals, Jul 22, 2017
    Last edited: Jul 25, 2017
  37. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

    any suggestions for learning to web scrape with python?
     
  38. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    No idea, but learn up on some regular expressions if you're trying to parse raw HTML.
     
    Joe_Pesci likes this.
  39. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    example of a script I used to scrape 538

    Code:
    import collections
    from lxml import html
    import csv
    import requests
    import datetime
    import pandas as pd
    import datetime
    
    
    URL = 'https://projects.fivethirtyeight.com/2017-mlb-predictions/games/'
    
    request = requests.get(URL)
    
    tree = html.fromstring(request.text)
    
    Away_teams = []
    for x in tree.xpath('//*[@id="js-index"]/main/div[1]/div[2]/section/div/div[1]/div/div/div[1]/p[1]/a[2]'):
        Away_teams.append(x.text.strip())
    Home_teams = []
    for x in tree.xpath('//*[@id="js-index"]/main/div[1]/div[2]/section/div/div[1]/div/div/div[1]/p[1]/a[3]'):
        Home_teams.append(x.text.strip())
    Home_Perc = []
    for x in tree.xpath('//*[@id="js-index"]/main/div[1]/div[2]/section/div/div[1]/div/div/div[3]/p'):
        Home_Perc.append(x.text.strip())
    Away_Perc = []
    for x in tree.xpath('//*[@id="js-index"]/main/div[1]/div[2]/section/div/div[1]/div/div/div[2]/p'):
        Away_Perc.append(x.text.strip())
    
    
    df = pd.DataFrame(
        {'Home': Home_teams,
         'Away': Away_teams,
         'Home_Perc': Home_Perc,
         'Away_Perc': Away_Perc,
         'Game_Date': str(datetime.datetime.now().date())
        })
    
    
    df = df[['Away', 'Home', 'Away_Perc', 'Home_Perc', 'Game_Date']]
    
    df.to_excel(str(datetime.date.today()) + '_MLB_Predictions.xlsx', index='False')
    
     
    Joe_Pesci and clemsontyger04 like this.
  40. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    Scrapy is also a good library to use

    some sites you have to use selenium to scrape, like if you need to login first before scraping, like this one

    Code:
    import scrapy
    import time
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    
    class BboSpider(scrapy.Spider):
        name = "bbo"
        allowed_domains = ["flipboard.com"]
        login_page = "http://www.flipboard.com/signin"
    
        def start_requests(self):
            driver = webdriver.PhantomJS()
            driver.get(self.login_page)
            input_path = "input[data-testid='sign-in-username']"
            pw_path = "input[data-testid='sign-in-password']"
            button_path = "button[data-testid='sign-in-signin-button']"
            time.sleep(5)
            driver.find_element_by_css_selector(input_path).send_keys('tmb')
            driver.find_element_by_css_selector(pw_path).send_keys("tmbhoss")
            driver.find_element_by_css_selector(button_path).click()
            time.sleep(5)
    
            print (driver.page_source)
            driver.close()
            yield scrapy.Request("http://www.flipboard.com")
    
        def parse(self, response):
            print('test')
            print (response.text)
            for data in response.css('div.item'):
                yield {
                    'subject': data.css('a::text').extract(),
                    'body': data.css('p::text').extract(),
                    'link': data.css('a::attr(href)').extract(),
                }
    
     
    Joe_Pesci and clemsontyger04 like this.
  41. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    when are the tmb programming broz gonna join together to work on a cool million dollar project?
     
  42. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    i started off using BeautifulSoup and it was pretty slick

    i think there are better options now, though, that was like 5 yrs ago
     
    Joe_Pesci likes this.
  43. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    when we get a million dollar idea

    i have a thousand-dollar idea, this football sim, but it's so overwhelmingly huge that my progress has slowed in the past year
     
    Joe_Pesci and colonelrascals like this.
  44. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

    scrapy apparently does a lot of the "under the hood" stuff for you, which was perfect for me since my project was basically just to research some language i had never used, write something simple that demos something it does well, and do a presentation

    i wanted to aggregate lines from multiple betting sites to start building a database for nfl and college football but i ended up having to scale the project back and now all it does is get the day's current baseball scores
     
    leroi likes this.
  45. Joe_Pesci

    Joe_Pesci lying dog-faced pony soldier
    Donor
    Wolfsburg

    nm
     
    #1047 Joe_Pesci, Aug 14, 2017
    Last edited: Aug 15, 2017
  46. Room 15

    Room 15 Mi equipo esta Los Tigres
    Donor
    Auburn TigersAtlanta BravesAtlanta FalconsUnited States Men's National Soccer Team

    Anybody used Python to load data into Google BQ? We're switching from Teradata so I'll need to move some of our jobs over
     
  47. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    come join me on the boulevard of broken dreams
     
    Joe_Pesci likes this.
  48. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    yes, if you count the old NDB client library that comes as part of appengine.

    i havent' tried to fuck with the newer stuff much